Numenta offers a wide range of pre-trained and optimized models for different use cases. If you need customization for your specific use case, you can fine-tune a model with the NuPIC Training Module.

Note: It's recommended to choose the pre-trained model based on your use case, accuracy and performance requirements, and available resources. Find tips on how to select the right model here.

Non-Generative Models

Model Name	Architecture	Description
hf-bert.base-1-v0	BERT	A unoptimized model from Hugging Face for benchmarking.
nupic-bert-2-v0	BERT	An optimized model for embedding texts while offering exceptional adaptability to different NLP tasks.
nupic-bert-1-v0	BERT	A smaller version of `nupic-bert-2-v0`, ideal for real-time applications.
nupic-sbert-2-v1	SBERT	An optimized model that excels in sentence-level tasks.
nupic-sbert-1-v3	SBERT	A smaller, much faster version of `numenta-sbert-2-v1`. Ideal for real-time applications.
nupic-sbert-2-v2	MosaicBERT	An optimized model that excels in sentence-level tasks with higher pre-training and finetuning accuracy.
nupic-sbert-1-v5	MosaicBERT	A smaller, much faster version of `numenta-sbert-2-v2`. Ideal for real-time applications.

Non-Generative Architectures and Use Cases

BERT

BERT (Bidirectional Encoder Representations from Transformers) is used for a wide variety of general-purpose language tasks. If you need a versatile model for various NLP tasks, BERT is a great choice.

Example use cases: sentiment analysis, question-answering, text classification, named entity recognition (NER)

Example applications: customer feedback analysis, automated chatbot responses, location detection

You can find example code that leverages our BERT models for sentiment analysis here.

SBERT

SBERT (Sentence-BERT) modifies the BERT architecture and is used to compare the meaning of sentences and texts, making them ideal for sentence-level tasks. If you need to find documents or passages with similar meanings in a large corpus, SBERT is a powerful tool.

Example use cases: sentence/document analysis, semantic similarity, information retrieval

Example applications: document clustering, contract analysis, recommendation systems, plagiarism detection

You can find example code that leverages our SBERT models for sentence similarity here.

MosaicBERT

MosaicBERT is a custom BERT architecture optimized for faster training and higher accuracy compared to BERT. For domain-specific tasks or tasks that require high training/fine-tuning efficiency, MosaicBERT models are a great place to start.

Generative Models

Model Name	Description
nupic-gpt.zs-7b-v0	NuPIC's default generative AI model, specifically optimized using neuroscience principles.
gemma2.it-2b-v0	A smaller, faster generative AI model.
llama-7b-v0	A generative AI model designed for broad NLP applications.
zephyr-7b-v0	A generative AI model with high efficiency and adaptability in specific domain tasks.

Generative Architectures and Use Cases

Generative models are useful for a wide variety of natural language tasks, and can also provide in-context learning and few-shot capabilities. NuPIC includes instruction-tuned generative models that can provide chat functionality.

Example use cases: semantic understanding, contextual summarization, dialogue generation

Example applications: summarization for news articles, assistant / chatbot, document and knowledge retrieval

NuPIC-GPT

NuPIC-GPT is our own generative AI model, optimized using neuroscience principles. It performs well on CPUs while being more accurate than Gemma.

Gemma

Gemma is a group of lightweight large language models that share technology with the larger Gemini models. Gemma's smaller footprint allows it to run more efficiently and/or in resource-constrained environments.

Llama 2

Llama 2 is a series of large language models offered as both base foundation models and instruction-tuned “chat” models.

Zephyr

Zephyr is a fine-tuned version of Mistral-7B-v0.1 that was trained on on a mix of publicly available, synthetic datasets using Direct Preference Optimization (DPO).

Model Sizing

The size of a model can affect its performance, speed, and resource requirements.

Size 1 (e.g. nupic-bert-1-v0): These BERT-base equivalent models relatively lightweight, and are therefore faster to run and require less compute. They are ideal for real-time applications, tasks that require low-latency inference, or deployment on edge.
Size 2 (e.g. nupic-bert-2-v0):These models are roughly twice as large as their Size 1 counterparts. These models, with their increased capacity, can understand more complex structures and nuances in data, which typically comes with an increase in accuracy.
LLMs sizes (e.g. llama-7b-v0): Large language models have their sizes expressed in billions of parameters in their names. Performance, accuracy and compute characteristics of large language also vary with size, as explained above.

Tokenizers

With Tokenizer (-wtokenizer): Each model comes with an integrated tokenizer, making them able to accept raw text inputs.
Without Tokenizer: You can use the models without a tokenizer. You will need to tokenize the text data separately in the correct format before using them.

Fine-Tuning

Can't find what you need from our Model Library? You can use one of them as a starting point to fine-tune a model with your own data!