Tutorial

After installation, you are ready to start testing the convenience and power of the package.

Before using, type

>>> import shorttext

You will get the message that Theano, Tensorflow or CNTK backend is used for keras. Refer to Keras Backend for information about switching backends.

• Data Preparation
  • Example Training Data 1: Subject Keywords
  • Example Training Data 2: NIH RePORT
  • Example Training Data 3: Inaugural Addresses
  • User-Provided Training Data
• Text Preprocessing
  • Standard Preprocessor
  • Customized Text Preprocessor
  • Tokenization
  • Reference
• Document-Term Matrix
  • Preparing for the Corpus
  • Using Class DocumentTermMatrix
  • Reference
• Character to One-Hot Vector
  • Reference
• Supervised Classification with Topics as Features
  • Topic Vectors as Intermediate Feature Vectors
  • Topic Models in gensim: LDA, LSI, and Random Projections
  • AutoEncoder
  • Abstract Latent Topic Modeling Class
  • Classification Using Cosine Similarity
  • Classification Using Scikit-Learn Classifiers
  • Notes about Text Preprocessing
  • Reference
• Word Embedding Models
  • Word2Vec
  • GloVe
  • FastText
  • Poincaré Embeddings
  • BERT
  • Other Functions
  • Links
  • Reference
• Word-Embedding Cosine Similarity Classifier
  • Sum of Embedded Vectors
  • Reference
• Deep Neural Networks with Word-Embedding
  • Wrapper for Neural Networks for Word-Embedding Vectors
  • Provided Neural Networks
  • Reference
• Maximum Entropy (MaxEnt) Classifier
  • Maxent
  • Reference
• Character-Based Sequence-to-Sequence (seq2seq) Models
  • Creating One-hot Vectors
  • Training
  • Decoding
  • Model I/O
  • Reference
• Stacked Generalization
  • Reference
• Metrics
  • Edit Distance and Soft Jaccard Score
  • Word Mover’s Distance
  • Jaccard Index Due to Cosine Distances
  • BERTScore
  • Reference
• Spell Correctors
  • Norvig
  • Sakaguchi (SCRNN - semi-character recurrent neural network)
  • Reference

Home: Homepage of shorttext