Skip to end of metadata
Go to start of metadata

Log Level Data - Protocol Buffers

AppNexus Log-Level Data (LLD) is encoded using a binary interchange format called Protocol Buffers. Use the steps below to get started with protobuf and integrate LLD into your system. 

Protobuf installation and configuration

When subscribing to feeds using the Log Level Data Service, you may choose to receive your data in one of the available protobuf formats (highly recommended). The protobuf format allows for more streamlined schema changes and convenient ingestion of the data into big data systems. 

The procedure below provides an example project that includes all feed schemas and demonstrates how to read AppNexus protobuf files. The example project includes example binary files as well as source code that can be used as a starting point for ingestion.

  1. Review the following Install dependencies. Be sure to complete the relevant tasks.
    1. Install the protobuf compiler. Use one of the following methods:
      • Use a package manager using one of the following commands:
        • For OSX, use the command: brew install protobuf250

        • For Ubuntu, use the command: sudo apt-get install protobuf-compiler

      • Build from source: https://github.com/google/protobuf/tree/v2.5.0

    2. If you will be using protobuf format (not protobuf-delimited), install hadoop-client with native Snappy support.
    3. If you plan to use the examples project, install Maven. Maven is available via most package managers.
  2. Download the example project (get the latest one from the File Formats and Schemas section of Batch Log-Level Data) and unzip it. 

  3. Refer to the README file in the example project for further directions.

  • No labels