Apache OpenNLP · JSON Structure

Apache Opennlp Tokenization Result Structure

TokenizationResult schema from Apache OpenNLP

Type: object Properties: 3
Machine LearningNatural Language ProcessingNLPText ProcessingApacheOpen SourceJava

TokenizationResult is a JSON Structure definition published by Apache OpenNLP, describing 3 properties. It conforms to the https://json-structure.org/meta/core/v0/# meta-schema.

Properties

tokens spans probabilities

Meta-schema: https://json-structure.org/meta/core/v0/#

JSON Structure

Raw ↑ 
{
  "$schema": "https://json-structure.org/meta/core/v0/#",
  "$id": "https://raw.githubusercontent.com/api-evangelist/apache-opennlp/refs/heads/main/json-structure/apache-opennlp-tokenization-result-structure.json",
  "description": "TokenizationResult schema from Apache OpenNLP",
  "type": "object",
  "properties": {
    "tokens": {
      "type": "array",
      "items": {
        "type": "string"
      },
      "description": "Extracted tokens",
      "example": [
        "Pierre",
        "Vinken",
        ",",
        "61",
        "years",
        "old"
      ]
    },
    "spans": {
      "type": "array",
      "items": {
        "$ref": "#/components/schemas/Span"
      }
    },
    "probabilities": {
      "type": "array",
      "items": {
        "type": "double"
      },
      "description": "Confidence for each token boundary"
    }
  },
  "name": "TokenizationResult"
}