Batch Segment Service
The Batch Segment Service allows you to upload audiences to the AppNexus platform through a batch/bulk upload framework. This data can be used in conjunction with data from buyers or sellers for the purposes of campaign targeting or yield management. All data sent through the Batch Segment Service is appended to the existing segment data already in our system.
- Ability to upload compressed files
- Error checking of segment data
- Configurable input file format
- Confirmation of successful upload
- Feedback on overall processing status
- Association of segments to users regardless of location of users
- A high maximum data volume
For optimal results, it is strongly recommended implementing the best practices found Batch Segment Service Best Practices.
The Batch Segment Service requires configuration prior to use. Please consult Batch Segment Service - Configuration to learn how to configure it for your seat.
Gzip is the only file compression method supported by this service.
Add a Segment File for Processing
Adding your segment file to the system is a three-step process. First, request a job identification number and upload URL. Second, upload your file to the assigned upload URL. Third, check the job's processing status. Please note that for now, files are limited to at most 1800 segments on any individual line. If you have more than 1800 segments for one user you must break that line into multiple lines.
- Step 1. Request an upload URL and job ID
- Step 2. Post the file to the upload URL
- Step 3. Check the job status
Step 1. Request an upload URL and job ID
Each segment data file that is uploaded must be associated with a particular job ID. This ID is used to create the upload URL and to track the file's processing status. The first step is to send an empty
POST request to the service.
Step 2. Post the file to the upload URL
The file upload URL is given in the JSON response to Step 1 by the field
upload_url. You must
POST your segment file to this URL for processing. You'll receive a JSON object that tells you if the upload succeeded. Do not hardcode the upload URL in your application - make sure to dynamically grab it from the upload_url field.
You must begin your upload to the given Upload URL within five (5) minutes, and only one URL is valid at any given time. If you wait longer than 5 minutes to start your upload you must request a new URL.
We recommend you do not exceed one upload per minute at the least. If you have more than 200 jobs waiting to be processed at any given time, you will be prohibited from uploading additional jobs.
Your segment file should not be larger than 0.5GB.
In order for the file to upload correctly, you must specify the MIME type in the HTTP header as "Content-Type: application/octet-stream". Do not use "Content-Type: application/x-www-form-urlencode" (-d or --data flags in curl). Using an incorrect MIME type will prevent the file from being processed by the API Batch Segment Service.
Your file must conform to the Latin1 character set.
Step 3. Check the job status
Finally, check the processing status by sending a
GET request. The JSON response contains information such as how long the file took to process and the number of errors, if any. Note that you should wait until phase="completed" before looking at the results fields such as
num_valid. For more detailed information, see JSON Fields below.
Per AppNexus SLA, allow up to 24 hours for the file to process.
If you are a data provider using the Impbus API, note that the
batch_segment_upload_job field will be an array with a single object inside of it, e.g.
Possible Upload Errors
The following are errors that may happen when:
- You've canceled the upload
- The upload phase exceeds 90 minutes
- You've reached one of its four upload limits (daily bytes, hourly bytes, or hourly lines limit)
Possible Processing Errors
If the value of the
num_invalid_format field is greater than
"0", check the values in the
error_log_lines field (see Example below). Note the following in the
num_invalid_formatindicates there was a problem parsing a line in the uploaded file.
"failed with an illegal number of fields"indicates that the number of fields in a
segment_fieldsblock did not match what was defined in the batch-segment config (see Batch Segment Service - Configuration for more information). In the below example, the config indicates three fields:
but the parser only found two fields:
View Your File Upload History
To see metadata about all of your segment file uploads within the last 30 days, make a
GET call to the service with your
member_id specified in the query string. The JSON response will include an array of
batch_segment_upload_job objects. For more information about the specific fields of the
batch_segment_upload_job object, see JSON Fields.
File upload history is available for the last 30 days only.
Note that our API limits responses to 100 objects via pagination. You can view additional objects by appending one of these to the API call:
You can read more about pagination on our wiki here.
To find out which fields you can filter and sort by, make a GET call to https://api.appnexus.com/batch-segment/meta.
This is the ID of the
An automatically generated number.
The status of the API call; successful calls return
The object whose fields contain metadata describing the upload and processing job. If you are using the Impbus API, this will be an array containing a single object. See Batch Segment Upload Job for details.
Batch Segment Upload Job
When you request the status of your processing job, the system returns a
batch_segment_upload_job object (if you are a data provider, this will be an array containing a single object). Depending upon which request you're making to the service, it will contain some or all of the following metadata (For more information about the required sequence of requests, see Add a Segment File for Processing). Note that most metadata will only be present when phase = "completed."
The URL where you'll upload your segment data file.
The current processing status, one of "error", "starting", "uploading", "validating", "processing", or "completed."
The time at which file upload began.
The time at which the file associated with this job ID was uploaded.
The time at which file validation was completed.
The time at which file processing was completed.
If phase='error', this error code describes the type of error encountered. Note that an error code will only be shown here if there was an error with the uploading, validating, or processing of the file itself (i.e. does not include invalid format or invalid segment errors). Common errors are caused by unreadable files and exceeding defined object limits. Returns
How long it took to process the segment file, in minutes.
The percentage of the processing that has been completed, given the current
The number of valid lines in the uploaded file. Each user/segment combination is considered 1 line.
The number of uploaded lines containing formatting errors (This depends upon your particular file format configuration). Duplicate lines will also be considered to be an invalid format.
This is a count of unique input lines that have a valid user ID.
This is a count of unique input lines that have an invalid or nonexistent user
The number of invalid segments in the file. Deduplicated.
The number of invalid timestamps in the file.
The number of segments in the file which you are unauthorized to access. Deduplicated.
The number of expired segments in the file. Deduplicated.
The number of inactive segments in the file. Deduplicated.
This is a placeholder value not currently in use.
A string containing newline-separated lines. Each line lists a validation error or the reason for an error while uploading your file. You can choose how many lines (200 by default) appear in this field.
A string containing newline-separated lines. Each line lists a segment and how many users were successfully added to it. This field defaults to 200 lines.
The unique identifier of this object.
A string of alphanumeric characters that uniquely identifies the processing job associated with this file.
Your member ID.
The creation date of this object.
The most recent modification date of this object (usually via