Large-Language-Models on The Edge Cases

Anthropic Streaming Event Types

Tue, 28 Jan 2025 03:57:36 +0000

Anthropic model streaming responses are delivered in event chunks. Below is a list of the event types with some notes on what they mean.

Event Types

event: message_start
data: {"type": "message_start", "message": {"id": "msg_1nZdL29xx5MUA1yADyHTEsnR8uuvGzszyY", "type": "message", "role": "assistant", "content": [], "model": "claude-3-5-sonnet-20241022", "stop_reason": null, "stop_sequence": null, "usage": {"input_tokens": 25, "output_tokens": 1}}}

This indicates the beginning of a message. It is notable for giving the model-provided count of the input tokens, as well as the initial output token count.

event: content_block_start
data: {"type": "content_block_start", "index": 0, "content_block": {"type": "text", "text": ""}}

The beginning of a block of response content, whether it’s text or something else.

LLM Knowledge Management Agent

Sat, 06 May 2023 16:47:59 +0000

Overview

Description: An agent to research a given topic by doing analytical and syntopic reading, summarizing and extracting main points, connecting them together, and forming new knowledge outputs such as zettels or evergreen notes.

Lessons Learned

~~Document embeddings with transformers don’t seem to be too practical. Instead, go with a sentence embedding for the knowledgebase using note titles and search with main idea embeddings.
Dispatch table values need to the the function as an object, so without the parentheses. Otherwise they’ll be run when instantiated.

Project Notes

Future Improvements

Future improvements to the project should go here for easy reference. They should be a type of project note and link back to a Project page and any tickets that come from it.