Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion config/_default/menus/main.en.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6844,7 +6844,7 @@ menu:
identifier: observability_pipelines_socket
weight: 421
- name: Splunk HEC
url: observability_pipelines/destinations/splunk_hec
url: observability_pipelines/destinations/splunk_hec/logs/
parent: observability_pipelines_destinations
identifier: observability_pipelines_splunk_hec
weight: 422
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ These are the available destinations:
[16]: /observability_pipelines/destinations/opensearch/
[17]: /observability_pipelines/destinations/sentinelone/
[18]: /observability_pipelines/destinations/socket/
[19]: /observability_pipelines/destinations/splunk_hec/
[19]: /observability_pipelines/destinations/splunk_hec/logs/
[20]: /observability_pipelines/destinations/sumo_logic_hosted_collector/
[21]: /observability_pipelines/destinations/syslog/
[22]: /observability_pipelines/destinations/amazon_s3/
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
title: Splunk HEC
type: multi-code-lang
external_redirect: /observability_pipelines/destinations/splunk_hec/logs/
---
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
---
title: Splunk HTTP Event Collector (HEC) Destination
description: Learn how to set up the Splunk HEC destination for logs in Observability Pipelines.
disable_toc: false
products:
- name: Logs
icon: logs
url: /observability_pipelines/configuration/?tab=logs#pipeline-types
aliases:
- /observability_pipelines/destinations/splunk_hec/
code_lang: logs
type: multi-code-lang
weight: 1
---

{{< product-availability >}}

## Overview

Use Observability Pipelines' Splunk HTTP Event Collector (HEC) destination to send logs to Splunk HEC.
Expand Down Expand Up @@ -63,13 +63,7 @@ Select the **Encoding** in the dropdown menu (**JSON** or **Raw**).
{{< tabs >}}
{{% tab "Secrets Management" %}}

- Splunk HEC token identifier:
- References the Splunk HEC token for the Splunk indexer.
- The default identifier is `DESTINATION_SPLUNK_HEC_TOKEN`.
- Splunk HEC endpoint URL identifier:
- References the Splunk HTTP Event Collector endpoint your Observability Pipelines Worker sends processed logs to. For example, `https://hec.splunkcloud.com:8088`.
- **Note**: `/services/collector/event` path is automatically appended to the endpoint.
- The default identifier is `DESTINATION_SPLUNK_HEC_ENDPOINT_URL`.
{{% observability_pipelines/splunk_hec_secrets %}}

{{% /tab %}}

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
---
title: Splunk HTTP Event Collector (HEC) Destination
description: Learn how to set up the Splunk HEC destination for metrics in Observability Pipelines.
disable_toc: false
code_lang: metrics
type: multi-code-lang
weight: 2
---

## Overview

Use Observability Pipelines' Splunk HTTP Event Collector (HEC) destination to send metrics to Splunk HEC.

## Setup

Configure the Splunk HEC destination when you [set up a pipeline][1]. You can set up a pipeline in the [UI][3], using the [API][4], or with [Terraform][5]. The steps in this section are configured in the UI.

**Notes**:
- The Splunk index you send your metrics to must be a metrics index. If you send them to an events index, you can't view the metrics in Splunk using any metrics type queries, such as `mcatalog` and `mstats`.
- If you don't attach your index to the Splunk authentication token you are using for Observability Pipelines, you must enter the name of the [index](#splunk-index) when you set up the destination.

<div class="alert alert-danger">For Secrets Management: Only enter the identifiers for the Splunk HEC token, endpoint, and if applicable, the TLS key pass. Do <b>not</b> enter the actual values.</div>

{{% observability_pipelines/secrets_env_var_note %}}

After you select the Splunk HEC destination in the pipeline UI:

1. Enter the identifier for your token. If you leave it blank, the [default](#secret-defaults) is used.
1. Enter the identifier for your endpoint URL. If you leave it blank, the [default](#secret-defaults) is used.

### Optional settings

#### Default namespace

Enter a default namespace to use as a prefix for metrics that don't already have one. The namespace is prepended to the metric name with a period (`.`), such as `service.cpu.usage` where `service` is the namespace.

#### Compression

If you want to compress your metrics with gzip, select **gzip** in the dropdown menu. The default compression is **None**.

#### Splunk index

Enter the name of the Splunk metrics index to which you are sending metrics. See [template syntax][6] if you want to route metrics to different indexes based on specific fields in your metrics.

**Note**: This **Index** field is only optional if you have an index attached to the Splunk authentication token you are using for Observability Pipelines. Otherwise, you must enter the name of the Splunk metrics index.

#### Source

Enter a source value to add the source field to your metrics.

#### Source type override

Set the `sourcetype` to override Splunk's default value, which is `httpevent` for HEC data. See [template syntax][6] if you want to route metrics to different source types based on specific fields in your metrics.

#### Buffering

{{% observability_pipelines/destination_buffer %}}

#### Enable TLS

{{% observability_pipelines/tls_settings %}}

## Secret defaults

{{% observability_pipelines/set_secrets_intro %}}

{{< tabs >}}
{{% tab "Secrets Management" %}}

{{% observability_pipelines/splunk_hec_secrets %}}

{{% /tab %}}

{{% tab "Environment Variables" %}}

{{% observability_pipelines/configure_existing_pipelines/destination_env_vars/splunk_hec %}}

{{% /tab %}}
{{< /tabs >}}

## How the destination works

### Event batching

A batch of events is flushed when one of these parameters is met. See [event batching][2] for more information.

| Maximum Events | Maximum Size (MB) | Timeout (seconds) |
|----------------|-------------------|-------------------|
| None | 1 | 1 |

[1]: /observability_pipelines/configuration/set_up_pipelines/
[2]: /observability_pipelines/destinations/#event-batching
[3]: https://app.datadoghq.com/observability-pipelines
[4]: /api/latest/observability-pipelines/
[5]: https://registry.terraform.io/providers/datadog/datadog/latest/docs/resources/observability_pipeline
[6]: /observability_pipelines/destinations/#template-syntax
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@
- The Splunk HEC token for the Splunk indexer. **Note**: Depending on your shell and environment, you may not want to wrap your environment variable in quotes.
- The default environment variable is `DD_OP_DESTINATION_SPLUNK_HEC_TOKEN`.
- Base URL of the Splunk instance:
- The Splunk HTTP Event Collector endpoint your Observability Pipelines Worker sends processed logs to. For example, `https://hec.splunkcloud.com:8088`.
- The Splunk HTTP Event Collector endpoint your Observability Pipelines Worker sends processed data to. For example, `https://hec.splunkcloud.com:8088`.
**Note**: `/services/collector/event` path is automatically appended to the endpoint.
- The default environment variable is `DD_OP_DESTINATION_SPLUNK_HEC_ENDPOINT_URL`.
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
- Splunk HEC token identifier:
- References the Splunk HEC token for the Splunk indexer.
- The default identifier is `DESTINATION_SPLUNK_HEC_TOKEN`.
- Splunk HEC endpoint URL identifier:
- References the Splunk HTTP Event Collector endpoint your Observability Pipelines Worker sends processed data to. For example, `https://hec.splunkcloud.com:8088`.
- **Note**: `/services/collector/event` path is automatically appended to the endpoint.
- The default identifier is `DESTINATION_SPLUNK_HEC_ENDPOINT_URL`.