Is access to cloud storage via API limited?

[pdxguy]

I have a script that automatically backs up my videos to Dropbox: https://github.com/janick/ArloDownload

Recently, I have seen it hang and crash after normally processing a certain number of files:

urllib3.exceptions.ProtocolError: ('Connection aborted.', BadStatusLine("''",))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/janick/ArloData/ArloDownload.py", line 301, in <module>
    thisHelper.getLibrary(thisHelper.cameraLibs[camera])
  File "/home/janick/ArloData/ArloDownload.py", line 230, in getLibrary
    response = self.session.get(url, stream=True)

Is NetGear throttling access to their cloud storage? 

[jguerdat]

I use the same basic script, just not the janick one.  He modified it for his purposes so he probably should be the one to talk to.  I haven't had that error but it may be related to my script stopping before all videos are downloaded on occasion.  I simply rerun the script to ensure I've gotten all videos.  Netgear has never throttled me - I believe that either something in the script isn't handled well or there's a break in the network communications for some reason.  It's beyond my capability to debug so I simply rerun the script.

[pdxguy]

I'm Janick 😉

[jguerdat]

Hah! Hope your debug skills are better than mine.

If you rerun the script, does it pick up where it left off? That's what happens to me.

[pdxguy]

Yes it does. Goes through a bunch of video files (including the ones that it choked on earlier) then hits another snag.

[pdxguy]

And my debug skills are only as good my my Google-fu and stackoverflow.com 😉 According to  https://stackoverflow.com/questions/25326616/unexpected-keyword-argument-buffering-python-client, it comes from an aborted connection on the server side.

Traceback (most recent call last):
  File "/home/janick/ArloData/ArloDownload.py", line 301, in <module>
    thisHelper.getLibrary(thisHelper.cameraLibs[camera])
  File "/home/janick/ArloData/ArloDownload.py", line 225, in getLibrary
    self.concatenate(library[idx:startIdx])
  File "/home/janick/ArloData/ArloDownload.py", line 255, in concatenate
    response = self.session.get(url, stream=True)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 469, in get
    return self.request('GET', url, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 457, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 569, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 407, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', BadStatusLine("''",))

[jguerdat]

There have been a few times when I had to run the script 3-4 times to complete. Usially only 1-2 times is necessary.