Machine-Learning on The Edge Cases

Model Context Protocol Clients Make Use of MCP Servers to Get External Tools, Prompts, Resources, and More

Thu, 30 Jan 2025 21:12:48 +0000

Purpose

Clients serve as the intermediary between MCP servers and hosts (LLMs).

Development Pattern

Client Class

Model Context Protocol Servers Define and Serve Functions That Can Extend Client Functionality Using an LLM For Coordination

Thu, 30 Jan 2025 20:42:30 +0000

Development Pattern

Use uv to initialize a new project
Create venv with uv venv
Add mcp and other dependencies
In Pyproject, add build system and script entry point
__init__.py should have a main() function that runs the server

Tools

Implement a list_tools() handler that lists available tools
Implement any helper functions that do business logic
Implement call_tool() handler. Takes name str and arguments dict, name will correspond to name of tool in list_tools() handler
Implement a main function to run the server using packages from mcp.server

Prompts

Prompt architecture

Anthropic Streaming Event Types

Tue, 28 Jan 2025 03:57:36 +0000

Anthropic model streaming responses are delivered in event chunks. Below is a list of the event types with some notes on what they mean.

Event Types

event: message_start
data: {"type": "message_start", "message": {"id": "msg_1nZdL29xx5MUA1yADyHTEsnR8uuvGzszyY", "type": "message", "role": "assistant", "content": [], "model": "claude-3-5-sonnet-20241022", "stop_reason": null, "stop_sequence": null, "usage": {"input_tokens": 25, "output_tokens": 1}}}

This indicates the beginning of a message. It is notable for giving the model-provided count of the input tokens, as well as the initial output token count.

event: content_block_start
data: {"type": "content_block_start", "index": 0, "content_block": {"type": "text", "text": ""}}

The beginning of a block of response content, whether it’s text or something else.

"Double free or corruption" Error Opening Images with Pillow

Thu, 22 Feb 2024 16:23:22 +0000

If an image isn’t properly closed after opening with Pillow, it’s likely because you’re trying to pass the opened image itself without operating on it. Images are lazily loaded until they’re operated on, which can cause multiple being opened to overlap each other in physical memory. To solve this, load images and operate on them one at a time.

Source

Pillow docs

Heads in Machine Learning Are Prediction Components of a Deep Neural Network

Mon, 18 Dec 2023 02:34:00 +0000

In deep learning networks, the head or heads of such a network is the “top” of the network (as opposed to the “bottom” that ingests data) that does some predictive task (such as a classification head or regression head). This is a separate component from the backbone network that does everything to produce the component on which the head or heads completes its tasks (in CNNs, this is the final convolutional feature map of the input image).

LLM Knowledge Management Agent

Sat, 06 May 2023 16:47:59 +0000

Overview

Description: An agent to research a given topic by doing analytical and syntopic reading, summarizing and extracting main points, connecting them together, and forming new knowledge outputs such as zettels or evergreen notes.

Lessons Learned

~~Document embeddings with transformers don’t seem to be too practical. Instead, go with a sentence embedding for the knowledgebase using note titles and search with main idea embeddings.
Dispatch table values need to the the function as an object, so without the parentheses. Otherwise they’ll be run when instantiated.

Project Notes

Future Improvements

Future improvements to the project should go here for easy reference. They should be a type of project note and link back to a Project page and any tickets that come from it.

Data Parallelism

Mon, 17 Apr 2023 20:58:24 +0000

[same: Model Parallelism]

Data parallelism is when you split the data and train the full model in parallel on each segment of data. This is different from Model Parallelism in which the model itself is split across different GPUs.

Machine Learning System Design Interview

Sat, 15 Apr 2023 02:43:54 +0000

Note: See here for valid statuses

[author: Ali Aminian Alex Xu]

Summary

What are the key ideas? How can I apply this knowledge that I learned? How do these ideas relate to what I already know?

💡Ideas

Note: These should become Project Idea fleeting notes with a link back to this source.

Fleeting Notes

Note: Conceptual notes that aren’t as tied to the source material. These are fodder for processing into evergreen notes.

Task Machine Learning Engineers and Platform MLEs

Sun, 18 Dec 2022 03:31:19 +0000

Note

Task MLEs work with data scientists to “productionize” models, sustaining a specific pipeline or pipelines in production. They need to write, maintain, and monitor pipelines that are responsible for the entire lifecycle of a ML model.

Platform MLEs help task MLEs automate their job. They build pipelines that support multiple tasks (task MLEs solf specific tasks).

Platform MLEs build pipelines to create features, task MLEs create pipelines to use features. PMLEs trigger ML performance drop alerts, TMLEs act on alerts.

Real World Problem Spaces Exhibit Non-Stationarity ie They Drift

Thu, 24 Mar 2022 22:32:56 +0000

Data Drift

Machine learning models are stationary. The trained model is an artifact that doesn’t move, however the problem space in the real world does (see this diagram), which is referred to as data drift. Recall that machine learning models can be thought of as existing in a multidimensional Euclidian space.

There are two kinds of drift:

concept drift - also known as real drift, this is a shift in the relationship between model inputs and outputs. Because p(y | x) changes, this always causes model decay.
data drift - also known as input drift, feature drift, or covariate shift, this is a change to the statistical distributions within the input data. It can cause model decay by driving a change in p(y | x) but doesn’t necessarily. When it doesn’t, it is referred to as virtual drift.

Both kinds of drift can manifest in at least 4 modes: 1. Abrupt 2. Incremental 3. Gradual 4. Reoccurring (e.g. seasonality)

Supervised Machine Learning Data Exists in a Euclidean Space

Thu, 24 Mar 2022 15:32:29 +0000

All training data exists as points in a multidimensional Euclidean space. You can visualize an enclosure of this data (the training domain). Within that space, supervised ML acts as interpolation geometry.

Visualization

In reality, the training domain will look jagged and weird because data is messy, incomplete, and often doesn’t line up with the regional problem space. Because ML models can not extrapolate, the model won’t work well at the edges or outside of the training domain.

FourthBrain - ML Model Drift and Decay

Tue, 22 Mar 2022 21:43:08 +0000

FourthBrain - ML Model Drift and Decay

Metadata

URL: https://www.youtube.com/watch?v=C1yiOonTYqM Author: FourthBrain

Literature Notes

TABLE rows.file.link AS "Literature Note", rows.file.cday AS "Date"
FROM #note/literature AND Example Title
GROUP BY file.link
sort date ASCENDING

Note: there may not be many literature notes for this, because it’s a how-to type of talk, where there won’t be notes on the piece itself, but on what it teaches.

Fleeting Notes

Supervised ML is interpolation geometry. “All training data exists as points in a Euclidean space…we can create an enclosure. Then, inside that enclosure, we can interpolate.” Global problem space -> regional problem space -> training domain -> model. When we get data, there are quantity and quality gaps which are smaller than the ideal training domain. This means that a model won’t work well at trying to predict on observations that fall outside the training domain because machine learning models for non-linear problems are interpolation machines. They can not extrapolate. Active learning can fix gaps between the training domain and regional problem space. This flags and prioritizes confusing inputs for human ground-truthing and augments the training set with newly labeled observations to fill gaps in the training domain. Most machine learning methods are based on the assumption that the problem space does not change over time. In the real world, though, the problem space does indeed change - this is called non-stationarity. In other words, the problem space drifts out of alignment with the existing training domain and model (data drift). Addressing data drift can take the form of filling gaps with active learning or pruning old or obsolete observations by using a weighting scheme (among other strategies). MLOps and Pipelines are beneficial for monitoring models for appreciable drift and refitting the model on augmented training data when drift is detected. Two kinds of drift: - concept drift - or real drift; a shift in the relationship between model inputs and outputs (changes p(y | x)). This always causes model decay. - data drift - change to the statistical distributions within the input data (also known as input drift, feature drift, covariate shift). This can cause model decay by driving a change in p(y | x). If it doesn’t cause that change, then it’s called virtual drift. Both kinds of drift can manifest in 4 modes: 1. Abrupt 2. Incremental 3. Gradual 4. Reoccurring (like seasonality)

Database Engineering and Machine Learning Systems for Data Science Curriculum

Fri, 18 Feb 2022 00:47:25 +0000

This is a list of resources to gain an advanced understanding of databases and machine learning systems that data engineers, MLOps professionals, ML engineers, and other titles build for R&D groups. Because this is a major part of my own job description, I wanted to compile this list of courses to be consumed as projects to improve what I build for my own teams.

The first iteration of this comes completely from Arun Kumar’s response thread to me asking about how to learn these things without shelling out for an MS in CS (still a goal of mine, though!).

Machine-Learning on The Edge Cases

Model Context Protocol Clients Make Use of MCP Servers to Get External Tools, Prompts, Resources, and More

Purpose

Development Pattern

Client Class

See also

Model Context Protocol Servers Define and Serve Functions That Can Extend Client Functionality Using an LLM For Coordination

Development Pattern

Tools

Prompts

Anthropic Streaming Event Types

Event Types

"Double free or corruption" Error Opening Images with Pillow

Source

Heads in Machine Learning Are Prediction Components of a Deep Neural Network

LLM Knowledge Management Agent

Overview

Lessons Learned

Project Notes

Future Improvements

Data Parallelism

Machine Learning System Design Interview

Summary

💡Ideas

Fleeting Notes

Task Machine Learning Engineers and Platform MLEs

Note

Real World Problem Spaces Exhibit Non-Stationarity ie They Drift

Data Drift

Supervised Machine Learning Data Exists in a Euclidean Space

Visualization

FourthBrain - ML Model Drift and Decay

Metadata

Literature Notes

Fleeting Notes

Database Engineering and Machine Learning Systems for Data Science Curriculum