Llama is a family of large language models (LLMs) developed by Meta. These models are trained on massive collections of text and can generate human-like responses, answer questions, summarize documents, and perform many other language tasks. Like other neural networks, Llama models consist of learned parameters (weights) that are obtained during training.
Llama.cpp¶
Llama.cpp is an open-source inference engine written in C++ for performing inference of large language models (LLMs). While frameworks such as scikit-learn and PyTorch are primarily used to train machine-learning models, llama.cpp is designed to efficiently execute already trained language models on CPUs and consumer hardware. It supports compressed GGUF model files, enabling models with billions of parameters to run on modern laptops.
In this course, llama.cpp is used to explore the capabilities and limitations of large language models, including text generation, question answering, and chemistry-specific applications. Running models locally allows students to experiment with LLMs without requiring cloud-based services or specialized hardware.
Llama-cpp-python¶
The llama-cpp-python package provides a Python interface to the llama.cpp inference engine, allowing large language models (LLMs) stored in the GGUF format to be executed locally from Python programs and Jupyter notebooks. It handles prompt submission, text generation, and token streaming.