Looker API Performance Best Practices

[Diego_Solorzano]

This post is focused on the Looker API 4.0. As of Looker 22.4, version 3.1 is deprecated. You can read the official documentation here.

Here are some best practices to improve the API performance calls when developing with the Looker API 4.0. It will take into consideration the usage of the official SDKs, specifically the Typescript version for examples (But can be applied to any other official SDK).

1. The initialization of the SDK authentication is only needed once per session.
   • Per session, once we run the init40 method for the SDK we do not need to re-create it for each request.

     const sdk = LookerNodeSDK.init40();

2. Endpoints that return all values of an object type should be avoided when we have a large instance (high number of Dashboards, Folders, Looks, and so on).
   • Some requests to endpoints like the GetAllDashboards or GetAllLooks will take some time to load if there is a high number of those objects in the instance.
   • The reason for this is because Looker will check for permissions and metadata for each object and run several internal DB queries that will take time to complete. Also, these endpoints will return all objects of the type from all users.
   • The solution/workaround for this is to use the /search endpoints for each object. For example, the SearchDashboards and SearchLooks.
     
     Those endpoints have filtering and pagination functionalities. Which will greatly reduce the time for the request to complete.
      
   • Example: You would like to display the user's Dashboards in your custom Web Application.

     const search_params: IRequestSearchDashboards = {
       user_id: "USER_ID",
       limit: 5,
       offset: 0,
     };
     const searched_dashboards = await sdk.ok(
       sdk.search_dashboards(search_params)
     );

     You would need to use the user_id to filter the specific User and use limit + offset to work with pagination.

3. Specify the fields to be returned to reduce parsing/querying of extra attributes.

   • ​​​​​​​Most of the endpoints that return an object or a list of them will have the fields attribute. It is strongly recommended to use it as there are some fields that call additional internal DB queries. Therefore, by not specifying those fields, the request time will improve.

   • Example: This first call gets the ID, Name, Parent ID, all dashboards, all looks, and permissions ('can') from a specific folder (ID 18).
     
     It will run slowly on larger instances because we are querying permissions for the folder ('can'), and for all of the dashboards and looks, we will get all of their attributes as well. This means that we would query each of their permissions and metadata as well. Resulting in a high number of internal DB queries.

     const my_folder = await sdk.ok(
       sdk.folder("18", "id,name,parent_id,dashboards,looks,can")
     );
     

     
     The solution to make this request run much faster, would be to specify even further the fields to be returned.
     
     In this second call, we removed the 'can' field to skip the permission query for the folder (If for your case there's the need to use that field, use it with caution as it increases request time). And we will specify fields for the Dashboards and Looks being returned as well.​​​​​​

     const my_folder = await sdk.ok(
       sdk.folder("18", "id,name,parent_id,dashboards(id,title),looks(id,title)")
     );
     

     
     This will result in a much smaller list of values returned. And all the calls for permissions would be avoided as well. Making the request run much faster. At the end we want to specify the fields that will be used in our Custom Application and ignore the others.

4. Make usage of the API Explorer to quickly troubleshoot loading times for multiple endpoints. ​​​​​​​

   • ​​​​​​​Looker's API Explorer is a great tool to install in your instance to troubleshoot different endpoints in a short amount of time. You won't need to setup any SDK authentication as it uses your current user's session to make the internal calls.