In this post, we will share best practices for downloading large data sets. Specifically, we will outline how to accomplish this using reports and why this is more efficient than paging with selectors.
When you request a large amount of data using generic selectors (for example, keywords), you may find that the responses are slow for very large data sets. First, we advise you not to page past 10,000 records due to the increased amount of time it takes to skip the first 10,000 records. Second, as a rule of thumb, once you exceed 100,000 keywords, it is a good idea to use a report to obtain your data rather than generic selectors.
Reports are designed to handle large quantities of data. In addition, you will also use fewer API units if you request data via reports than if you request the same data via the AdGroupCriterionService. For example, say you would like to obtain all the keywords across all ad groups using paging of 1000 for 100,000 total keywords. This would require 100,000 API units, one per AdGroupCriterion. Defining a report and downloading the data that way will cost significantly less units in the long run. If you define the report with a sliding date range such as
If you need fields that are not available in reports, you still have some options to increase performance. With generic selectors you can specify only the fields you need; the fewer the number of requested fields, the faster the results will be returned. You can also use predicates to filter and limit the amount of data in each result set. By requesting keywords on a per-campaign or per-ad group basis instead of all at once, you may actually get your results faster.
If you have any questions about using reports with the API, you can ask us on the forum.
- Kevin Winter, AdWords API Team
When you request a large amount of data using generic selectors (for example, keywords), you may find that the responses are slow for very large data sets. First, we advise you not to page past 10,000 records due to the increased amount of time it takes to skip the first 10,000 records. Second, as a rule of thumb, once you exceed 100,000 keywords, it is a good idea to use a report to obtain your data rather than generic selectors.
Reports are designed to handle large quantities of data. In addition, you will also use fewer API units if you request data via reports than if you request the same data via the AdGroupCriterionService. For example, say you would like to obtain all the keywords across all ad groups using paging of 1000 for 100,000 total keywords. This would require 100,000 API units, one per AdGroupCriterion. Defining a report and downloading the data that way will cost significantly less units in the long run. If you define the report with a sliding date range such as
TODAY or LAST_WEEK, you can reuse the report each time you need to download your data set. Please see this page for more information about reports.If you need fields that are not available in reports, you still have some options to increase performance. With generic selectors you can specify only the fields you need; the fewer the number of requested fields, the faster the results will be returned. You can also use predicates to filter and limit the amount of data in each result set. By requesting keywords on a per-campaign or per-ad group basis instead of all at once, you may actually get your results faster.
If you have any questions about using reports with the API, you can ask us on the forum.
- Kevin Winter, AdWords API Team
