Header & body capture

Capture request and response headers and bodies with OpenTelemetry (OTel) to investigate failures. If you only need response times and traces, start with the automatic eBPF Observer. Header and body capture requires language-specific instrumentation settings.

A span records one operation in a request's trace. Send spans over the OTLP protocol to the Docker Collector's gateway at http://everyup-agent:4318, which attributes them to the matching service and forwards them to Web, or directly to Web at /api/v1/otlp/v1/traces. For direct connections, see Monitoring setup. If the Docker Collector profile disables Docker discovery, each app must send its service-scoped bearer token; see Networking Notes.

Quickest path — the bundled instrumentation (Java, Node.js)

Docker Collector installation includes the everyup-otel helper and an OpenTelemetry bundle (Java agent jar and Node.js bootstrap). Automatic setup targets Java and Node.js services running under Compose.

Language	Header capture	Body capture
Java	Configured by the bundle	Requires a filter in application code
Node.js	Configured by the bundle	Off by default. Enable 요청·응답 바디도 수집 (also capture request/response bodies)

Supported versions are JVM 8+, Node.js 18.19+ or 20.6+. Applying the settings restarts the selected services. See the header and body examples below for Java body capture and manual setup in other languages.

Apply the settings

1. In Web, open the environment from Docker 환경 (Docker environments) and choose 상세 수집 설정 (detailed collection settings).
2. Tick the Compose project and the Java or Node.js services to change. Nothing is selected at first.
3. Enter the application Compose path and whether to capture bodies, then click 변경 사항 확인 (Review changes).
4. Check which services restart and which settings are added, then run the 서버 변경 미리보기 (server change preview) command.
5. When the real Compose validation passes, run the 안전 적용 명령 (safe apply command) and check 상세 수집 적용 결과 (apply results) in Web.

The preview command validates Compose and the restart scope without changing the app. The apply command writes a separate override beside the original Compose file and restarts only selected services. A failed verification restores the previous configuration automatically.

Confirm receipt and recover

Send requests to the app, then check 상세 수집 적용 결과 (apply results). Web shows 설정 적용 검증 완료 (configuration verified) separately from 실행 시작 이후 새 트레이스 수신 (new traces since the run started). Traces may arrive through other paths such as eBPF, so receipt alone does not prove header and body capture works. Inspect headers in the trace and, if you enabled body capture, inspect the captured bodies as an admin. A lack of traces due to no traffic does not trigger a rollback.

Run the following on the server with the application's Compose file to check status or restore the previous settings. The status and verify commands below check local state.

sudo everyup-otel status ./docker-compose.yml
sudo everyup-otel verify ./docker-compose.yml
sudo everyup-otel rollback ./docker-compose.yml

Run records and command behavior

everyup-otel plan ./docker-compose.yml --project=shop api=node validates running state, the Compose project, and generated settings without changing app settings, containers, volumes, or networks. A lock prevents concurrent CLI changes to the same Compose project.

Applying writes docker-compose.everyup.yml and verifies existing JAVA_TOOL_OPTIONS/NODE_OPTIONS, the bundle mount, Collector network, and container health. Settings for unselected services are preserved.

The apply command generated by Web includes --report=<Docker environment ID>/<run ID>, so the result is sent back to Web. A run plan must start within one hour of being issued, and a plan that already completed cannot be applied again. Reporting uses the Web address and API key stored in the server's everyup-agent container, so the key never appears in a command shown in the browser or in the run result. If Web does not approve the run start, the app is not changed. If reporting fails, check the CLI output.

Automatic and manual rollback results are recorded with the run. Recent runs remain visible when you reopen the screen.

What the helper does

Installation, verification, and recovery internals

The one-time Docker Collector installer also installs /usr/local/bin/everyup-otel. When you run the apply command, the helper:

• validates Linux, Docker, the base Compose file, target runtimes, and the currently running containers before changing anything;
• creates a shared everyup-monitoring network and populates the everyup-instrumentation volume from the Docker Collector image;
• preserves existing JAVA_TOOL_OPTIONS or NODE_OPTIONS and writes a managed docker-compose.everyup.yml next to the original Compose file;
• recreates only the selected services, then checks health, injection options, the read-only /everyup mount, and Collector network connectivity;
• automatically restores and recreates the previous configuration if restart or verification fails.

The helper keeps rollback metadata in the app Compose directory's .everyup folder. It attaches the app and Docker Collector to the shared everyup-monitoring network so everyup-agent:4318 resolves without publishing an OTLP port. Existing JAVA_TOOL_OPTIONS / NODE_OPTIONS values are retained and the EveryUp option is appended once.

Everything below is for other languages, manual SDK setups, or understanding the span contract the bundle produces.

What EveryUp reads from a span

To appear in a service's API 요청 (API requests) tab, emit a SERVER-kind span per request with:

Attribute	Type	Purpose
http.request.method (or http.method)	string	required — request method
http.response.status_code (or http.status_code)	int	required — status code
url.path (or http.target)	string	request path
http.route	string	optional path template (e.g. /users/:id)
client.address (or net.peer.ip)	string	optional client IP

A span missing method or status is ignored (not projected as an API request). Span duration becomes the request latency (sub-millisecond rounds up to 1ms).

Headers

Headers ride on the standard OTel span attributes http.request.header.<name> / http.response.header.<name> and show up in the Trace panel's Headers section. Capture is allowlist-based per language:

Language	How
Java (agent jar)	OTEL_INSTRUMENTATION_HTTP_SERVER_CAPTURE_REQUEST_HEADERS=content-type,user-agent (and ..._CAPTURE_RESPONSE_HEADERS)
Node (EveryUp bundle)	OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST=content-type,user-agent (and ..._SERVER_RESPONSE)
Python (opentelemetry-instrument)	OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST=content-type,user-agent
Other / manual SDK	set the attributes yourself on the SERVER span

Sensitive headers (authorization, cookie, set-cookie, x-api-key, ...) are masked at ingest regardless of what you capture — both dash and underscore spellings.

Request/response bodies (the event contract)

Bodies are not a standard span attribute, so they ride as span events on the request span:

• Event name request_body_masked and/or response_body_masked.
• Each event carries a body attribute holding the body text you have already masked — strip secrets, tokens, and PII in your instrumentation before export. Optional attributes: body_size (int), body_truncated (bool).
• Keep bodies small (the bundle default is 8KiB). Web enforces a 64KiB server-side cap and flags anything above it as truncated.

Captured bodies are admin-only: Web redacts the body attribute for non-admin users, records every admin view in audit_events, and strips the bodies after EVERYUP_RETENTION_BODYCAPTUREDAYS days (default 7); the traces stay.

Node.js

Covered by the EveryUp bundle — set EVERYUP_CAPTURE_BODIES=true (EVERYUP_BODY_MAX_BYTES, EVERYUP_MASKED_BODY_FIELDS to tune). No code.

Java (Spring Boot example)

The OTel Java agent does not capture bodies; add a small filter that feeds the current span:

import io.opentelemetry.api.common.AttributeKey;
import io.opentelemetry.api.common.Attributes;
import io.opentelemetry.api.trace.Span;
import org.springframework.web.util.ContentCachingRequestWrapper;
import org.springframework.web.util.ContentCachingResponseWrapper;

@Component
public class BodyCaptureFilter extends OncePerRequestFilter {
    private static final int MAX = 8192;
    private static final AttributeKey<String> BODY = AttributeKey.stringKey("body");

    @Override
    protected void doFilterInternal(HttpServletRequest req, HttpServletResponse res,
                                    FilterChain chain) throws ServletException, IOException {
        var reqW = new ContentCachingRequestWrapper(req, MAX);
        var resW = new ContentCachingResponseWrapper(res);
        try {
            chain.doFilter(reqW, resW);
        } finally {
            Span span = Span.current();
            span.addEvent("request_body_masked",
                Attributes.of(BODY, mask(new String(reqW.getContentAsByteArray(), StandardCharsets.UTF_8))));
            span.addEvent("response_body_masked",
                Attributes.of(BODY, mask(new String(resW.getContentAsByteArray(), StandardCharsets.UTF_8))));
            resW.copyBodyToResponse();
        }
    }

    // Mask secrets BEFORE export — extend to your own field names.
    private String mask(String body) {
        return body.replaceAll("(\"(?:password|token|secret|apiKey)\"\\s*:\\s*\")[^\"]*", "$1***");
    }
}

Python (FastAPI/Starlette example)

import json, re
from opentelemetry import trace

MASKED = re.compile(r'("(?:password|token|secret|api_key)"\s*:\s*")[^"]*')

@app.middleware("http")
async def capture_request_body(request, call_next):
    body = await request.body()  # Starlette caches it; handlers can still read it
    response = await call_next(request)
    span = trace.get_current_span()
    if span.is_recording() and body:
        text = body[:8192].decode("utf-8", "replace")
        span.add_event("request_body_masked", {"body": MASKED.sub(r"\1***", text)})
    return response

Response-body capture with streaming responses is more involved in Starlette; start with request bodies.

Correlating logs with a request

The Trace panel combines spans, API requests, and logs with the same trace ID. To associate your application logs with a request:

• Using OTLP logs: configure the SDK log exporter and logging integration. Logs emitted inside a request appear in its trace panel when exported with the trace ID.
• Using plain stdout logs: print the trace ID or a request_id propagated through the x-request-id header in each line so you can search for it in Logs. Printing an ID in the message alone does not attach the log to the trace panel automatically.

If your service already logs request or response bodies, search those logs by the shared ID to inspect the recorded content. A shared ID does not recover bodies that were never logged.

Double-counting

Handled automatically: while a service ships real spans through the Docker Collector gateway, the Docker Collector pauses its access-log synthetic spans for that service (it resumes ~10 minutes after real spans stop). Only apps that send OTLP directly to Web, bypassing the gateway, can still double-count — point them at the gateway instead.