Star us on GitHub
Star
Menu

Fluent Forward

Set up highlight.io log ingestion via Fluent Forward (fluentd / fluentbit protocol).
1
Setup fluentd / fluent bit ingest.

Route your fluentd / fluent bit to forward://otel.highlight.io:24224. Regardless of the way you are using fluentbit, configure the tag to highlight.project_id=YOUR_PROJECT_ID to route the logs to the given highlight project

bin/fluent-bit -i cpu -t highlight.project_id=YOUR_PROJECT_ID -o forward://otel.highlight.io:24224
Copy
2
Running the fluent agent.

You may be running a fluent agent locally or in docker. In that case, you would use the fluent-bit.conf

[INPUT] name tail tag <YOUR_PROJECT_ID> path /var/log/your_log_file.log path_key file_path [INPUT] name tail tag <YOUR_PROJECT_ID> path /var/log/nginx/another_log_file.txt path_key file_path [FILTER] Name record_modifier Match * Record hostname my-hostname [OUTPUT] Name forward Match * Host otel.highlight.io Port 24224
Copy
3
(Optional) Configure Fluent Forward over TLS.

If you want to transfer data over a secure TLS connection, change the [OUTPUT] to the following (using port 24284)

[OUTPUT] Name forward Match * Host otel.highlight.io Port 24284 tls on tls.verify on
Copy
4
Setting up for AWS ECS?

If you are setting up for AWS Elastic Container Services, check out our dedicated docs for AWS ECS. .

5
Verify your backend logs are being recorded.

Visit the highlight logs portal and check that backend logs are coming in.