topic Re: Is access to cloud storage via API limited? in Arlo Pro

Is access to cloud storage via API limited?

pdxguy — Tue, 26 Sep 2017 16:58:35 GMT

I have a script that automatically backs up my videos to Dropbox: https://github.com/janick/ArloDownload

Recently, I have seen it hang and crash after normally processing a certain number of files:

urllib3.exceptions.ProtocolError: ('Connection aborted.', BadStatusLine("''",))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/janick/ArloData/ArloDownload.py", line 301, in <module>
    thisHelper.getLibrary(thisHelper.cameraLibs[camera])
  File "/home/janick/ArloData/ArloDownload.py", line 230, in getLibrary
    response = self.session.get(url, stream=True)

Is NetGear throttling access to their cloud storage?

Re: Is access to cloud storage via API limited?

jguerdat — Tue, 26 Sep 2017 17:06:26 GMT

I use the same basic script, just not the janick one. He modified it for his purposes so he probably should be the one to talk to. I haven't had that error but it may be related to my script stopping before all videos are downloaded on occasion. I simply rerun the script to ensure I've gotten all videos. Netgear has never throttled me - I believe that either something in the script isn't handled well or there's a break in the network communications for some reason. It's beyond my capability to debug so I simply rerun the script.

Re: Is access to cloud storage via API limited?

pdxguy — Tue, 26 Sep 2017 17:24:09 GMT

I'm Janick 😉

Re: Is access to cloud storage via API limited?

jguerdat — Tue, 26 Sep 2017 19:24:17 GMT

Hah! Hope your debug skills are better than mine.

If you rerun the script, does it pick up where it left off? That's what happens to me.

Re: Is access to cloud storage via API limited?

pdxguy — Tue, 26 Sep 2017 19:46:45 GMT

Yes it does. Goes through a bunch of video files (including the ones that it choked on earlier) then hits another snag.

Re: Is access to cloud storage via API limited?

pdxguy — Tue, 26 Sep 2017 19:49:28 GMT

And my debug skills are only as good my my Google-fu and stackoverflow.com 😉 According to https://stackoverflow.com/questions/25326616/unexpected-keyword-argument-buffering-python-client, it comes from an aborted connection on the server side.

Traceback (most recent call last):
  File "/home/janick/ArloData/ArloDownload.py", line 301, in <module>
    thisHelper.getLibrary(thisHelper.cameraLibs[camera])
  File "/home/janick/ArloData/ArloDownload.py", line 225, in getLibrary
    self.concatenate(library[idx:startIdx])
  File "/home/janick/ArloData/ArloDownload.py", line 255, in concatenate
    response = self.session.get(url, stream=True)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 469, in get
    return self.request('GET', url, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 457, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 569, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 407, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', BadStatusLine("''",))

Re: Is access to cloud storage via API limited?

jguerdat — Tue, 26 Sep 2017 23:11:51 GMT

There have been a few times when I had to run the script 3-4 times to complete. Usially only 1-2 times is necessary.