Writing-Projects on The Edge Cases

TypedDicts in Python

Thu, 13 Jan 2022 17:11:28 +0000

In Python 3.8+, you can create a dictionary with type annotations with the TypedDict class in the typing module. This is a regular dictionary that allows you to define it with type hints, allowing for better documentation of simple objects like dictionaries. To use them, create a class that inherits from typing.TypedDict and define your expected keys and their types like so:

from typing import TypedDict

class ModelMetadata(TypedDict):
    audience_id: str
    training_correlation: List[float]
    validation_correlation: List[float]
    final_sample_count: int
    final_accounts: List[AccountMetadata]

Setting Pynamo Model's Table After Instantiation

Mon, 22 Nov 2021 03:35:57 +0000

In some cases you may not know the table for a Pynamo model until you instantiate it. You can’t do this when you construct the object, but you can create a static method that takes the model class and table name and sets model.Meta.table_name to the table_name parameter, changing the table that model is connected to. This is useful if you set up models in one module and then import it elsewhere, and the table names come from user input.

Input and Output Channels for Sagemaker Training Jobs

Wed, 27 Oct 2021 19:08:01 +0000

Inputs

With SageMaker training jobs, you specify an “input channel” and configure it in the job definition. Each channel is a dictionary that specifies the data split. Each of these items will be made available on the instance in /opt/ml/input/data/<stage>.

JSON Example

{
 "train": {
 "ContentType": "trainingContentType",
 "TrainingInputMode": "File",
 "S3DistributionType": "FullyReplicated",
 "RecordWrapperType": "None"
 },
 "evaluation": {
 "ContentType": "evalContentType",
 "TrainingInputMode": "File",
 "S3DistributionType": "FullyReplicated",
 "RecordWrapperType": "None"
 },
 "validation": {
 "TrainingInputMode": "File",
 "S3DistributionType": "FullyReplicated",
 "RecordWrapperType": "None"
 }
}

CDK Example

Source

Outputs

With SageMaker training jobs, model artifacts should be saved to /opt/ml/model/ so that, with the training job output configuration, they will be zipped and added to S3. Failures can be written to the file /opt/ml/output/failure and the first 1024 characters will be returned as FailureReason.