The Iceberg REST Catalog API

Time: ~9 minutes. Tangible win: exchange client credentials for a bearer token, create a catalog, and create a namespace — using nothing but curl, no query engine involved.

Every engine integration in the next lesson is this same API underneath a driver. Seeing the raw HTTP calls first means Spark's catalog config later reads as "the engine doing this for me" instead of magic.

Primary source RFC 6749 §4.4 — OAuth 2.0 Client Credentials Grant, the auth flow used against /api/catalog/v1/oauth/tokens.

Two URL prefixes, two concerns

Catalog administration (creating catalogs, principals, roles, grants) lives under /api/management/v1/.... Actual Iceberg data-plane operations (namespaces, tables) live under /api/catalog/v1/{catalog}/... — two different path prefixes for two different concerns, both behind the same bearer token.

Creating a catalog requires a storageConfigInfo block naming a storageType (FILE for local disk/dev, S3/AZURE/GCS for production) and an allowedLocations allowlist.

The allowlist is a guardrail, not a bug Polaris will refuse to let tables be created outside allowedLocations. A mismatch here surfaces as a storage-validation error at table-create time, not at catalog-create time — which makes it look unrelated to how the catalog was configured if you don't already know to check this.

Exercise

# 1. Get a token with the root credentials from Lesson 3
curl -s -X POST http://localhost:8181/api/catalog/v1/oauth/tokens \
  -d 'grant_type=client_credentials&client_id=f103ea289b7df858&client_secret=<root-secret>&scope=PRINCIPAL_ROLE:ALL'

# 2. Create a catalog backed by local disk (swap TOKEN in for the access_token above)
curl -s -i -X POST -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  http://localhost:8181/api/management/v1/catalogs \
  -d '{"name": "coursecatalog", "type": "INTERNAL",
       "properties": {"default-base-location": "file:///data"},
       "storageConfigInfo": {"storageType": "FILE", "allowedLocations": ["file:///data"]}}'

# 3. Create a namespace inside it via the data-plane API
curl -s -i -X POST -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  http://localhost:8181/api/catalog/v1/coursecatalog/namespaces \
  -d '{"namespace": ["course_db"]}'

Verified output from a real local run:

Step 2 -> HTTP/1.1 201 Created
Step 3 -> HTTP/1.1 200 OK
           {"namespace":["course_db"],"properties":{"location":"file:///data/course_db"}}

Note the namespace response: Polaris computed file:///data/course_db by combining the catalog's default-base-location with the namespace name — that path was never specified directly.

Retrieval check

You need to create a new namespace. Which URL prefix does that call live under?

Correct. Namespaces and tables are Iceberg data-plane objects, which live under /api/catalog/v1/{catalog}/... — management/v1 is for admin objects like catalogs, principals, and grants.

Not quite. Data-plane objects (namespaces, tables) and admin objects (catalogs, principals, grants) live under different prefixes — mixing them up produces a 404, not a permissions error.

Your bearer token was issued 90 minutes ago with the default 3600-second expiry. A request now returns 401. What's the most likely cause?

Correct. Tokens expire (3600s by default here); a script that hardcodes one and outlives that window will start seeing 401s that have nothing to do with the catalog itself.

Not quite. A 401 at 90 minutes on a token issued with a 3600-second (60-minute) expiry is exactly what an expired token looks like — check the clock before suspecting the catalog.

Practice

  1. Run all three calls against your own server with your own catalog/namespace names.
  2. GET the namespace back and confirm the location Polaris derived for you.
  3. Deliberately set allowedLocations to a path that doesn't cover where you try to create a table, and observe where in the flow the error actually surfaces.
Ask the agent: "What JSON body would I send to list all namespaces instead of creating one — and does that call need a body at all?" Next lesson does the same operations through a real query engine instead of raw curl.