Duplicates and Missing Videos - YouTube Data API V3

[stefanodesaraca]

Hi,
I'd like to report some big issues I had while using YouTube Data API V3.
I used Python as programming language and two different methods to retrieve the videos' data.
The first method is executing the API requests and accessing the responses with the "requests" library.
The second one was executing the the requests and retrieving the results with the "google-api-python-client library".
For simplicity I will refer to the video's data with just "video".
For video's data I mean the four categories of data I tried to fetch with the API: "snippet, statistics, contentDetails, topicDetails".

1. Missing Videos in Channel when Trying to Fetch All of Them
My goal was to fetch all videos from a given YT channel, I already knew about the maximum return of 500 videos' data, so I adapted my program to iterate for a sufficient number of times to fetch all of the videos of the channel.
When I tried it I had no luck in obtaining every video's data, in fact, only half of them were actually returned.
Thinking I did something wrong trying with the requests library I tried with the "google-api-python-client library", but didn't manage to make it again and not all videos from the channel were returned.

All of this time I was wondering why it was so difficult to retrieve all videos' data from the channel, I did everyhing right... correct?
Yes, because I followed step by step the instructions of every single video/tutorial/documentation I found online.
But then I discovered the second problem.

2. Duplicate Videos Returned in the API Responses
Turns out that what I was getting from the API responses with both the first and second method were sometimes even duplicates, so the same video was present twice (or maybe more) in the same API response.
I verified that by putting al video IDs into a list and then converting it to a set.

That was too strange to be true, I had two problems which no one reported online, until I found this answer online:
https://stackoverflow.com/questions/65894790/how-to-avoid-omissions-in-video-information-acquisition-when-using-the-youtube-d/65899758#65899758
It worked, but still the "Search" endpoint doesn't work, and that's something that I would like to suggest to be repaired.
Then I discovered the third problem.

3. Different JSON Response Structure Between API Endpoints
Basically the dictionary structure of JSON responses is different between the endpoints, so for instance: if you use the "Search" endpoint you'll retrieve a JSON which has a specific structure, and if you use the "playlistItems" one you'll fetch a different JSON structure.
This is something that I didn't understand at first a bit because I didn't found any documentation which included an explenation of this change.

3.1 Missing Documentation of Uploads Playlist
While doing research online about possible solution I didn't find anything that documented the existance of the "Uploads Playlist" which contains all videos uploaded from the YouTube channel.
This is something that I think should be well documented on the API website.

I hope that this problems will be solved and I'm available for any question.
Thank you and have a great day.