Amazon SageMaker 101: Notebooks, Training Jobs, and Endpoints

This is the first post in our Amazon SageMaker series. Later posts cover training with built-in algorithms, deployment endpoints, Pipelines, and cost optimization.

Amazon Bedrock, covered in our previous series, solves the problem of calling an existing, pre-trained foundation model. SageMaker solves a genuinely different problem: training your own model — on your own data, for a task a general-purpose foundation model may not directly address at all — and deploying it, without personally provisioning and managing the underlying training and serving infrastructure by hand.

Notebooks: where exploration actually happens

import sagemaker
from sagemaker import get_execution_role
 
session = sagemaker.Session()
role = get_execution_role()
bucket = session.default_bucket()
 
print(f"Using bucket: {bucket}")
print(f"Execution role: {role}")

A SageMaker Notebook Instance is a managed Jupyter environment, pre-configured with the SageMaker SDK and appropriate IAM permissions already wired in through get_execution_role() — the same IAM-first design principle covered throughout this blog for GitHub Actions and Bedrock, applied here to a notebook's own access to S3, training infrastructure, and model artifacts. This is genuinely where data exploration, feature engineering, and initial model experimentation happen — not where production training or serving actually runs at real scale.

Training jobs: provisioned infrastructure, used, then released

from sagemaker.estimator import Estimator
 
estimator = Estimator(
    image_uri="683313688378.dkr.ecr.eu-west-2.amazonaws.com/sagemaker-xgboost:1.7-1",
    role=role,
    instance_count=1,
    instance_type="ml.m5.xlarge",
    output_path=f"s3://{bucket}/model-output",
)
 
estimator.set_hyperparameters(
    objective="binary:logistic",
    num_round=100,
    max_depth=5,
)
 
estimator.fit({"train": f"s3://{bucket}/training-data/train.csv"})

estimator.fit() is where the actual infrastructure provisioning happens — SageMaker spins up the requested instance(s), runs the training container against the specified S3 data, writes the resulting model artifact back to S3, and tears the training instance down automatically once the job completes. This is directly analogous to the disposable, ephemeral compute model covered for Docker containers and GitHub Actions runners earlier in this blog: you're billed only for the actual training duration, not for infrastructure sitting idle before or after — no persistent training server to separately provision, monitor, or remember to shut down.

Monitoring a training job while it runs

import boto3
 
client = boto3.client("sagemaker")
description = client.describe_training_job(TrainingJobName=estimator.latest_training_job.name)
print(description["TrainingJobStatus"], description["SecondaryStatus"])

InProgress Training

Training jobs run asynchronously — fit() blocks by default while it waits, but the underlying job (and its full logs, streamed to CloudWatch Logs automatically) can be inspected independently at any point, which matters for a genuinely long-running job where you don't want a notebook connection blocking the entire duration, or where a second person needs to check progress without access to the original notebook session.

Endpoints: deploying a trained model for real-time inference

predictor = estimator.deploy(
    initial_instance_count=1,
    instance_type="ml.m5.large",
)
 
result = predictor.predict([[5.1, 3.5, 1.4, 0.2]])
print(result)

Unlike a training job, deploy() provisions infrastructure that stays running — a real-time endpoint, ready to serve prediction requests continuously, billed for as long as it's up, not just for the duration of a single request. This is a meaningfully different cost and operational model than training: a training job's cost is bounded and predictable (a fixed job duration); an endpoint's cost accrues continuously until you explicitly tear it down.

Cleaning up: the step it's genuinely easy to forget

predictor.delete_endpoint()

An idle SageMaker endpoint left running after an experiment is finished accrues real, ongoing cost — the exact same "forgotten instance quietly costing money for months" risk flagged for a stray Terraform-provisioned resource earlier in this blog, applied here to inference endpoints specifically, which are especially easy to spin up during experimentation and then simply forget about once attention moves elsewhere.

Working locally against SageMaker resources

Beyond a hosted notebook, the SageMaker Python SDK works identically from a local machine or CI environment, as long as IAM credentials and permissions are configured correctly — genuinely useful for teams who prefer their own local development environment over a hosted notebook instance:

import sagemaker
import boto3
 
boto_session = boto3.Session(profile_name="ml-team", region_name="eu-west-2")
session = sagemaker.Session(boto_session=boto_session)

This is the same "the SDK and IAM model work the same regardless of where the code runs" principle covered for Bedrock in our previous series — a training job kicked off from a local script, a CI pipeline, or a hosted notebook all provision and behave identically, since the actual training infrastructure is the same either way.

The three core concepts, and how they relate

• Notebooks are for exploration and experimentation — not where production training or serving happens.
• Training jobs provision infrastructure for a bounded duration, then release it automatically — the disposable-compute model applied to model training specifically.
• Endpoints provision infrastructure that stays running continuously, for real-time inference — a genuinely different cost and operational profile than a training job, and something that needs to be deliberately torn down when no longer needed.

Studio: a fuller IDE beyond a single notebook instance

Beyond a standalone Notebook Instance, SageMaker Studio provides a fuller integrated environment — a single web-based IDE covering notebooks, experiment tracking, pipeline visualization, and model monitoring in one interface, rather than separate tools stitched together manually:

# Studio notebooks use the same SageMaker SDK identically —
# no code changes required to move from a Notebook Instance to Studio
session = sagemaker.Session()
role = get_execution_role()

The underlying SDK and IAM model are identical either way — Studio's real value is consolidating experiment tracking (comparing metrics across many training runs), pipeline visualization (seeing a Pipeline's DAG, covered later in this series, rendered directly), and model registry browsing into one place, rather than switching between the console, CloudWatch, and a notebook separately. For a team running many experiments and comparing results across them regularly, that consolidation is worth the switch; for occasional, simple training jobs, a standalone Notebook Instance remains entirely sufficient.

Instance types: matching compute to the actual workload

Just as a Terraform-provisioned EC2 instance should be sized to its actual workload rather than defaulting to whatever a tutorial happened to use, a SageMaker instance type deserves the same deliberate choice:

estimator = Estimator(
    image_uri=xgb_image,
    role=role,
    instance_count=1,
    instance_type="ml.m5.xlarge",   # general-purpose CPU
    # instance_type="ml.p3.2xlarge", # GPU-accelerated, for deep learning workloads
)

ml.m5 instances are general-purpose CPU compute, appropriate for classical ML algorithms like XGBoost or linear models. ml.p3 and ml.g5 instances provide GPU acceleration, genuinely necessary for training deep learning models (image classification, large neural networks) where CPU training would be impractically slow — but wasted spend for a workload that doesn't actually use a GPU at all. Choosing between them isn't a stylistic preference; it's the same instance-type-to-workload matching principle covered in depth in the cost-optimization post later in this series, applied here at the very first decision point in a project.

Lifecycle configurations: automating notebook setup

A Notebook Instance can run a lifecycle configuration script automatically on creation or every start — installing extra packages, cloning a repository — so a new team member's environment doesn't depend on remembering a manual setup checklist:

#!/bin/bash
set -e
pip install --upgrade pandas scikit-learn
git clone https://github.com/your-org/ml-notebooks.git /home/ec2-user/SageMaker/ml-notebooks

Attaching this as an "on-create" lifecycle configuration means every new Notebook Instance starts pre-configured identically — the same "codify the setup instead of relying on a person following a checklist correctly" principle behind a Dockerfile or an Ansible role, applied here to notebook provisioning.

What to actually remember from this post

• SageMaker trains and deploys your own model; Bedrock calls an existing foundation model — genuinely different problems, not competing solutions to the same one.
• Notebooks are for exploration; real training and serving run on separately-provisioned, purpose-specific infrastructure, and the same SDK works identically from a local machine or CI pipeline.
• A training job's infrastructure is automatically released when the job completes — bounded, predictable cost, the same disposable-compute principle as a CI runner or a Docker container — and can be monitored asynchronously via CloudWatch Logs.
• An endpoint stays running until explicitly deleted — a continuously-accruing cost that's genuinely easy to forget about after an experiment ends.

Next in the series: Training Your First Model with SageMaker Built-in Algorithms, where we go deeper on the training step introduced here.