Making Outgoing HTTP Requests from Effect Golem Agents

Use the Effect v4 HTTP client from effect/http. FetchHttpClient.layer supplies the canonical HttpClient.HttpClient service through QuickJS’s WASI-backed globalThis.fetch, while keeping request construction, failures, response decoding, and dependency provision in Effect.


import { Effect, Schema } from "effect";
import {
  FetchHttpClient,
  HttpClient,
  HttpClientRequest,
  HttpClientResponse,
} from "effect/http";

Do not wrap globalThis.fetch in Effect.tryPromise for ordinary application HTTP. Do not use node:http, node:https, @effect/platform-node, or another Node networking transport.


┌───────────────┐    HttpClient request    ┌───────────────────────┐    WASI HTTP    ┌────────────┐
│ Agent handler │─────────────────────────▶│ FetchHttpClient.layer │────────────────▶│ Remote API │
└───────────────┘                          └───────────────────────┘                 └────────────┘

The Http namespace from @golemcloud/effect-golem is separate: Http.mount, Http.get, and Http.post declare incoming agent routes. They do not execute outbound requests.

Provide the Fetch Client

Provide FetchHttpClient.layer at the narrowest common application boundary. A one-off request can provide it directly:


const readText = HttpClient.get("https://api.example.com/health").pipe(
  Effect.flatMap(HttpClientResponse.filterStatusOk),
  Effect.flatMap((response) => response.text),
  Effect.provide(FetchHttpClient.layer),
);

For agent handlers, provide the layer to the composed HTTP effect returned by the handler. If a larger application has several HTTP-dependent services, compose them into an application layer and provide FetchHttpClient.layer once at that layer boundary.

GET and Decode JSON

Build immutable requests, execute them through the service, require a 2xx status, and decode the JSON body with Effect Schema:


const ApiUser = Schema.Struct({
  id: Schema.Number,
  name: Schema.String,
});
 
const getUser = (id: number) =>
  HttpClientRequest.get(`https://api.example.com/users/${id}`).pipe(
    HttpClientRequest.setHeaders({
      Accept: "application/json",
      Authorization: "Bearer my-token",
    }),
    HttpClient.execute,
    Effect.flatMap(HttpClientResponse.filterStatusOk),
    Effect.flatMap(HttpClientResponse.schemaBodyJson(ApiUser)),
    Effect.provide(FetchHttpClient.layer),
  );

HttpClientResponse.schemaBodyJson both parses JSON and validates its shape. Transport, status, body-decoding, and schema failures remain in the Effect error channel.

For several calls, filter the client once:


const program = Effect.gen(function* () {
  const client = (yield* HttpClient.HttpClient).pipe(HttpClient.filterStatusOk);
  const response = yield* client.get("https://api.example.com/users/1", {
    headers: { Accept: "application/json" },
  });
  return yield* HttpClientResponse.schemaBodyJson(ApiUser)(response);
}).pipe(Effect.provide(FetchHttpClient.layer));

Choose either client-level HttpClient.filterStatusOk or response-level HttpClientResponse.filterStatusOk; do not apply both to the same request.

POST JSON

Use HttpClientRequest.bodyJson so serialization and the JSON content type are handled by the Effect HTTP request model:


const createUser = (name: string, email: string) =>
  HttpClientRequest.post("https://api.example.com/users").pipe(
    HttpClientRequest.setHeader("Accept", "application/json"),
    HttpClientRequest.bodyJson({ name, email }),
    Effect.flatMap(HttpClient.execute),
    Effect.flatMap(HttpClientResponse.filterStatusOk),
    Effect.flatMap((response) => response.text),
    Effect.provide(FetchHttpClient.layer),
  );

bodyJson is effectful because JSON encoding can fail. When a request body has an Effect Schema, prefer HttpClientRequest.schemaBodyJson(InputSchema)(input). Other request helpers include bodyText, bodyUint8Array, bodyUrlParams, bodyFormData, and bodyStream.

Use HttpClientRequest.put, patch, or delete for the corresponding HTTP method.

Read Responses

Status and headers are immediate response fields. Body readers are Effects:


const inspectResponse = (url: string) =>
  Effect.gen(function* () {
    const response = yield* HttpClient.get(url);
    const body = yield* response.text;
    return {
      status: response.status,
      contentType: response.headers["content-type"],
      body,
    };
  }).pipe(Effect.provide(FetchHttpClient.layer));

Use:

response.text for text;
response.json for an unvalidated parsed JSON value;
HttpClientResponse.schemaBodyJson(Schema) for validated JSON;
response.arrayBuffer for binary data; or
response.stream for streaming consumption.

Apply HttpClientResponse.filterStatusOk when non-2xx responses are failures. If the application must inspect a non-2xx body before deciding, read the response and branch on response.status instead.

Agent Handler Error Types

The HTTP client keeps transport, non-2xx, body-decoding, timeout, and schema failures typed in the Effect error channel. An Effect-Golem method with no error schema cannot return those failures. Either:

declare an Effect Schema for caller-visible failures and map the HTTP errors to that type; or
use Effect.orDie for genuinely unexpected failures that should become defects.


const infallibleHandlerEffect = getUser(1).pipe(Effect.orDie);

Do not use defects for expected business outcomes.

Timeouts and Cancellation

Put the timeout around request execution and body consumption, because receiving response headers does not mean the body is complete:


const timed = HttpClient.get("https://api.example.com/slow").pipe(
  Effect.flatMap(HttpClientResponse.filterStatusOk),
  Effect.flatMap((response) => response.text),
  Effect.timeout("5 seconds"),
  Effect.provide(FetchHttpClient.layer),
);

FetchHttpClient passes Effect interruption to a fetch AbortSignal. Prompt cancellation of the underlying WASI HTTP operation is currently blocked by the QuickJS runtime issue tracked in GOL-325. Do not claim that a timeout has already stopped the remote request until that runtime fix is consumed and verified.

Durability, Retries, and Duplicate Requests

Outgoing WASI HTTP calls are recorded by Golem. During replay of the same agent invocation, Golem reuses a completed call’s recorded response rather than sending the request again. Using FetchHttpClient.layer preserves that host behavior because it still executes through the same WASI-backed fetch implementation.

This is not cross-invocation deduplication. Two calls to the same agent method are independent and can send two requests. Recovery repairs an eligible incomplete call at its own Start; it is not a distributed transaction with the remote HTTP server.

For state-changing requests that callers may retry:

accept a stable caller-supplied request ID;
send it in the remote API’s idempotency header; and
require the remote API to deduplicate that key.

Do not assume Durability.generateIdempotencyKey deduplicates separate method invocations. Do not manually invoke a state-changing method when an automated verification will invoke it later.

Host status retry policy

Use a named host retry policy when retry depends only on HTTP status. The host applies it below FetchHttpClient and returns the final response to application code:


# golem.yaml — under retryPolicyDefaults / <environment>:
http-5xx-retry:
  priority: 20
  predicate:
    propIn: { property: "status-code", values: [500, 502, 503, 504] }
  policy:
    countBox:
      maxRetries: 3
      inner:
        exponential:
          baseDelay: "200ms"
          factor: 2.0

A retry can repeat a state-changing operation. Use it only when the operation is naturally idempotent or the remote server deduplicates a stable key.

Application-level validation

When retry depends on decoded content or several side effects, use explicit guest control flow or an SDK saga. Compensate successful steps before the next attempt. New attempts append new calls without deleting earlier responses or compensations. A defect after a response fails the invocation; it does not retry that completed request. Host transport and body-resumption retries remain host responsibilities.

Calling a Golem HTTP Endpoint

For ordinary structured bodies on an incoming Golem endpoint, user-supplied parameters not bound to path/query/header values come from a JSON object whose keys match the method’s input keys. Even one body parameter requires an object. A single object input named request needs { "request": { "text": "hello" } }, not the bare inner object. Unstructured text and binary inputs instead consume raw bodies.

Given an incoming method:


record: method({
  input: { message: Schema.String },
  success: Schema.Void,
  http: [Http.post("/record")],
});

send { "message": "..." }, not a raw string.

CLI invocation does not use this HTTP wrapper: pass one native TypeScript literal per user-supplied input in declaration order. Load golem-http-params-effect for HTTP mapping and golem-invoke-agent-effect for CLI examples and caller-only input rules.

Complete Agent Method

This durable agent performs the POST through one shared HttpClient service:


import { Effect, Schema } from "effect";
import {
  FetchHttpClient,
  HttpClient,
  HttpClientRequest,
  HttpClientResponse,
} from "effect/http";
import { defineAgent, method } from "@golemcloud/effect-golem";
 
export const MessageForwarder = defineAgent({
  name: "MessageForwarder",
  mode: "durable",
  id: { name: Schema.String },
  methods: {
    recordMessageViaHttp: method({
      input: { message: Schema.String },
      success: Schema.String,
    }),
  },
}).implement({
  init: () => Effect.void,
  methods: () => ({
    recordMessageViaHttp: ({ message }) =>
      HttpClientRequest.post(
        "http://test-app.localhost:9006/requests/main/record",
      ).pipe(
        HttpClientRequest.bodyJson({ message }),
        Effect.flatMap(HttpClient.execute),
        Effect.flatMap(HttpClientResponse.filterStatusOk),
        Effect.flatMap((response) => response.text),
        Effect.as(message),
        Effect.provide(FetchHttpClient.layer),
        Effect.orDie,
      ),
  }),
});

Import the implementation module from src/main.ts using its emitted .js suffix so the top-level registration runs.

For provider-backed LLM tasks, load golem-add-llm-effect and use LanguageModel plus the selected Effect AI provider instead of hand-writing provider REST calls.

Key Constraints

Import root Effect APIs from effect and HTTP APIs from effect/http.
Use the canonical HttpClient.HttpClient service with FetchHttpClient.layer.
Do not use direct globalThis.fetch, Node networking modules, or native HTTP client packages.
Keep handlers as Effects; do not return raw Promises or mark handlers async.
Use Effect request-body helpers instead of manually setting a fetch body.
Filter or inspect HTTP statuses explicitly and decode unknown JSON with Effect Schema.
Bound body consumption as well as request execution when applying a timeout.
Let Golem handle durable retries; do not add manual retry loops around host operations.
Durable replay does not deduplicate separate agent method invocations.
Use named camelCase JSON fields when posting to another Golem endpoint.