Introduction to LLMs

​

What is an LLM?

LLM stands for Large Language Model. Bolt generates code using Anthropic’s Claude 3.7 Sonnet and Claude 3.5 Sonnet LLMs, which provide robust programming performance. OpenAI’s ChatGPT is another popular LLM that you may have heard of or used yourself.

Without getting too detailed, LLMs work as a very smart autocomplete. It doesn’t ‘know’ information, but based on all of the training information it has been fed, it’s able to generate ‘new’ thoughts based on patterns that it detected during training.

In the case of Bolt, the LLM helps you generate code that aligns with your prompt, based on similar applications with similar specifications that it has been trained on.

​

Prompts

A prompt is a message you send to the AI. How you write your prompts is key to getting good results. Refer to Prompt effectively for more guidance.

​

Context

Context is all the information about the project that’s available to the AI: your prompts, previous responses, and the existing code. The context window is the amount of text that a large language model (LLM) can process at once.

Bolt provides a large context window, but you still can’t rely on the AI remembering your whole conversation. There’s a tradeoff here, as larger context windows consume tokens faster, which increases costs.

A way to preserve context while keeping the context window small is to get the AI to summarize the conversation so far, then reset the context window. Refer toReset the AI context window for instructions.

​

Tokens and LLMs

Understanding tokens and token usage is critical to using Bolt effectively.

A simple definition of tokens:

Tokens are small pieces of text

“I”, “love”, “cats”, ”!” are all examples of a token.

LLMs process text as tokens, analyze them, and then predict the next tokens to generate a response.

AI tokens are a complex topic related to all AI apps, not just Bolt. For detailed background on tokens in AI, check out Nvidia blog | Explaining Tokens — the Language and Currency of AI.

​

Token limits

LLMs can only handle a certain number of tokens at a time and the total includes both:

• The input you give (for example, a long question or document)
• The output it generates (for example, the response, or the code you get back)

​

Costs

If you’re using an LLM through a paid service (like you are with Bolt), costs are often calculated based on the number of tokens processed. Fewer tokens means lower cost.

In this table, you can find an approximate guide for estimating token costs for code tasks:

For Bolt specifically, it’s important to note that the majority of token usage is related to syncing your project’s file system to the AI: the larger the project, the more tokens used per message.

Costs can grow very quickly, so refer to Maximizing token efficiency to learn how to keep costs down.

Refer to Tokens for information on buying more tokens and token rollover.