It seems that you're using an outdated browser. Some things may not work as they should (or don't work at all).
We suggest you upgrade newer and better browser like: Chrome, Firefox, Internet Explorer or Opera

×
Community
General discussiongogrepo.py - Python script for regularly backing up your purchased GOG collection for full offline enjoyment(3896 posts)(3896 posts)
(3896 posts)
Pages:
1
50
100
150
200
222
223
224
225
226
260
This is my favourite topic
Posted October 23, 2022
avatar
Zaragon: I'm trying to run gogrepoc.py from my FreeBSD file server, and it's not working right for me. If I don't specify an os, it downloads what I'm guessing are the linux versions of games (giant shell scripts). If I specify "-os windows" it only downloads the extras. If I specify "-os windows -skipextras" it downloads nothing.

How can I can I convince it to actually download the windows installers? I don't game on the machine I'm downloading from and that seems to really be confusing the script.
If you've run an update on that machine then it's probably only recorded the information from the linux versions, so you'll need to run a full update specifying windows as well. If it's not that I'd recommend trying setting the language using -lang after that because my next guess would be that it's reporting a language that you own 0 games in.
Posted October 23, 2022
avatar
Zaragon: I'm trying to run gogrepoc.py from my FreeBSD file server, and it's not working right for me. If I don't specify an os, it downloads what I'm guessing are the linux versions of games (giant shell scripts). If I specify "-os windows" it only downloads the extras. If I specify "-os windows -skipextras" it downloads nothing.

How can I can I convince it to actually download the windows installers? I don't game on the machine I'm downloading from and that seems to really be confusing the script.
avatar
Kalanyr: If you've run an update on that machine then it's probably only recorded the information from the linux versions, so you'll need to run a full update specifying windows as well. If it's not that I'd recommend trying setting the language using -lang after that because my next guess would be that it's reporting a language that you own 0 games in.
That was the main problem, yes. I'll be honest, I didn't expect all of the automatic OS detection taking place behind the scenes, and it caught me off guard. (It also doesn't help that the script assumes any OS it doesn't recognize is linux.)

I had a hard time getting through the update because the script kept spitting errors. Two of them kept recurring over and over. First one is the XML parse error, which occurred 47 times during the update process, and looks like this:

13:00:32 | xml parsing error occurred trying to get md5 data for <file>
13:00:32 | The handled exception was:
13:00:32 |
Traceback (most recent call last):
File "./gogrepoc.py", line 713, in fetch_file_info
shelf_etree = xml.etree.ElementTree.fromstring(md5_response.content)
File "/usr/local/lib/python3.8/xml/etree/ElementTree.py", line 1321, in XML
return parser.close()
xml.etree.ElementTree.ParseError: no element found: line 1, column 0

Edit: I've traced down the cause for this issue. The script expects GOG to return either valid XML with the md5 value in it, or to return a 404 (not found) error code. The problem is that GOG is not doing either, it's doing this:

(Pdb) md5_response
<Response [200]>
(Pdb) md5_response.content
b''

So it's returning a 200 (OK) response code, but the page is completely empty. It's not so much that the XML is invalid as it is there simply isn't any XML at all.

The second is a distressingly long series of exceptions (roughly 224 lines each) that happened 90 times during the update, all for the same host, which all started with:

13:07:53 | request failed: HTTPSConnectionPool(host='gog-cdn-lumen.secure2.footprint.net', port=443): Read timed out. (3 retries left) -- will retry in 5s...

I can send the exception tracebacks to you directly, if it would be helpful.

And lastly, there's an issue when it goes to download, every single file fails to pre-allocate and spits an error "posix preallocation failed". The files seem like they're downloading okay anyway, so this error is mostly cosmetic, but I'd like to get to the bottom of why it's appearing. I'd also like to be able to just turn it off, since pre-allocation of anything isn't useful on ZFS (due to the fact that it's a copy-on-write filesystem).

Edit: posix_fallocate() isn't supported on ZFS+FreeBSD, at all; it returns EINVAL (I think). ZFS+Linux has a kludge that prevents the error from happening, but... it probably shouldn't. CoW filesystems (of which ZFS is only one) get no benefit from pre-allocating space due to their nature, anyway. I did find the spots in the code that were triggering the error and skip them if the OS detects as FreeBSD (detecting the filesystem the script is writing to is a significantly more complex undertaking and not worth it just for my own use).
Post edited October 24, 2022 by Zaragon
Posted October 24, 2022
avatar
LurkerLito: Anyone else have speed issues with this script? I was trying to use it to download Middle Earth Shadow of War which is like 153GB and it was trying to pull it 4 pieces at a time, but those 4 pieces were each only averaging about 8 MB/s. I tried using my browser and I got 3 pieces each running an average of 40 MB/s each with my 1 gbps connection at the same time as I was running gogrepo. So I restarted gogrepo but it still downloaded at the same slow speeds.
I'm not sure where the speed limit is, but what's interesting is you're reporting a different limit. I actually changed the number of threads from 4 to 8 to 16 on a hunch, and mine is limited to an overall throughput of about 18-20MB/s. So I get roughly 1.1MB/s with 16 download threads, 2.2MB/s with 8 download threads, 5MB/s with 4 download threads, etc. I actually get the best overall throughput with only two worker threads, as it will sometimes spike up to 25MB/s.
Posted October 24, 2022
avatar
Zaragon: I'm not sure where the speed limit is, but what's interesting is you're reporting a different limit. I actually changed the number of threads from 4 to 8 to 16 on a hunch, and mine is limited to an overall throughput of about 18-20MB/s. So I get roughly 1.1MB/s with 16 download threads, 2.2MB/s with 8 download threads, 5MB/s with 4 download threads, etc. I actually get the best overall throughput with only two worker threads, as it will sometimes spike up to 25MB/s.
Well I should also point out that the limit also depends also on where those files are located in the GoG content servers. For me at least Middle Earth Shadow of War files are located on fast servers. How I determine that is by how fast I can download a single file from my web browser. For example a single file from Shadow of War can go as burst as high as 100 MB/s but usually sustains an average of about 80 MB/s. But there are other games on GoG that if I download (like Hellish Quart) that seem to be on slow file servers and those generally average at most 16 - 20 MB/s.
Posted October 24, 2022
avatar
Zaragon: And lastly, there's an issue when it goes to download, every single file fails to pre-allocate and spits an error "posix preallocation failed". The files seem like they're downloading okay anyway, so this error is mostly cosmetic, but I'd like to get to the bottom of why it's appearing. I'd also like to be able to just turn it off, since pre-allocation of anything isn't useful on ZFS (due to the fact that it's a copy-on-write filesystem).

Edit: posix_fallocate() isn't supported on ZFS+FreeBSD, at all; it returns EINVAL (I think). ZFS+Linux has a kludge that prevents the error from happening, but... it probably shouldn't. CoW filesystems (of which ZFS is only one) get no benefit from pre-allocating space due to their nature, anyway. I did find the spots in the code that were triggering the error and skip them if the OS detects as FreeBSD (detecting the filesystem the script is writing to is a significantly more complex undertaking and not worth it just for my own use).
I have an issue with the gogrepoc.py preallocation as well when I am downloading on a Linux machine onto an external USB hard drive, which is NTFS-formatted (because I download only Windows-versions of games, even if I play them on Linux; but mostly I play them on Windows).

In my case the preallocation and download failed completely probably due to some timeout, not sure if this has been changed in the script. My only fix for that was to manually comment out the sections in the script, where it performs the preallocation. Maybe the issue is with Linux using ntfs-3g for writing to NTFS partitions and it is too slow to perform the preallocation etc.

I wish preallocation could be skipped altogether (I don't want to use it even in Windows; I will sometimes run a defragmenter, if really needed), but I guess I can manage by commenting out the preallocation myself...
Posted October 24, 2022
avatar
Zaragon: I'm not sure where the speed limit is, but what's interesting is you're reporting a different limit. I actually changed the number of threads from 4 to 8 to 16 on a hunch, and mine is limited to an overall throughput of about 18-20MB/s. So I get roughly 1.1MB/s with 16 download threads, 2.2MB/s with 8 download threads, 5MB/s with 4 download threads, etc. I actually get the best overall throughput with only two worker threads, as it will sometimes spike up to 25MB/s.
avatar
LurkerLito: Well I should also point out that the limit also depends also on where those files are located in the GoG content servers. For me at least Middle Earth Shadow of War files are located on fast servers. How I determine that is by how fast I can download a single file from my web browser. For example a single file from Shadow of War can go as burst as high as 100 MB/s but usually sustains an average of about 80 MB/s. But there are other games on GoG that if I download (like Hellish Quart) that seem to be on slow file servers and those generally average at most 16 - 20 MB/s.
Yes, I suspect the limit is primarily GOG side just because its *way* lower for Linux games than Windows and that's very consistent.
Posted October 24, 2022
avatar
Kalanyr: If you've run an update on that machine then it's probably only recorded the information from the linux versions, so you'll need to run a full update specifying windows as well. If it's not that I'd recommend trying setting the language using -lang after that because my next guess would be that it's reporting a language that you own 0 games in.
avatar
Zaragon: That was the main problem, yes. I'll be honest, I didn't expect all of the automatic OS detection taking place behind the scenes, and it caught me off guard. (It also doesn't help that the script assumes any OS it doesn't recognize is linux.)
It assumes linux because Windows and Mac actually have well defined detection signatures and those are checked first, and most devices that match neither are running something *Nix like enough for it to work.

avatar
Zaragon: I had a hard time getting through the update because the script kept spitting errors. Two of them kept recurring over and over. First one is the XML parse error, which occurred 47 times during the update process, and looks like this:

13:00:32 | xml parsing error occurred trying to get md5 data for <file>
13:00:32 | The handled exception was:
13:00:32 |
Traceback (most recent call last):
File "./gogrepoc.py", line 713, in fetch_file_info
shelf_etree = xml.etree.ElementTree.fromstring(md5_response.content)
File "/usr/local/lib/python3.8/xml/etree/ElementTree.py", line 1321, in XML
return parser.close()
xml.etree.ElementTree.ParseError: no element found: line 1, column 0

Edit: I've traced down the cause for this issue. The script expects GOG to return either valid XML with the md5 value in it, or to return a 404 (not found) error code. The problem is that GOG is not doing either, it's doing this:

(Pdb) md5_response
<Response [200]>
(Pdb) md5_response.content
b''

So it's returning a 200 (OK) response code, but the page is completely empty. It's not so much that the XML is invalid as it is there simply isn't any XML at all.
This is mainly sematics in that it the server reports that it's returning xml but is returning a blank file. Whether or not that's no XML or invalid XML in the form of an empty response is in the eye of the beholder.

avatar
Zaragon: The second is a distressingly long series of exceptions (roughly 224 lines each) that happened 90 times during the update, all for the same host, which all started with:

13:07:53 | request failed: HTTPSConnectionPool(host='gog-cdn-lumen.secure2.footprint.net', port=443): Read timed out. (3 retries left) -- will retry in 5s...

I can send the exception tracebacks to you directly, if it would be helpful.
Yeah, this happens because you get the full error stack for the 3 failed retries and the numerous abstraction layers it goes through to reach gogrepo. I should move these detailed breakdowns into debug priority and set logging to info by default .

avatar
Zaragon: And lastly, there's an issue when it goes to download, every single file fails to pre-allocate and spits an error "posix preallocation failed". The files seem like they're downloading okay anyway, so this error is mostly cosmetic, but I'd like to get to the bottom of why it's appearing. I'd also like to be able to just turn it off, since pre-allocation of anything isn't useful on ZFS (due to the fact that it's a copy-on-write filesystem).

Edit: posix_fallocate() isn't supported on ZFS+FreeBSD, at all; it returns EINVAL (I think). ZFS+Linux has a kludge that prevents the error from happening, but... it probably shouldn't. CoW filesystems (of which ZFS is only one) get no benefit from pre-allocating space due to their nature, anyway. I did find the spots in the code that were triggering the error and skip them if the OS detects as FreeBSD (detecting the filesystem the script is writing to is a significantly more complex undertaking and not worth it just for my own use).
It happens because I haven't done enough research yet to know, how to ask for the underlying file system on the target device specifically in python.

And that there's no real cost beyond some mild annoyance to posix_fallocate failing. It just does a normal allocation after detecthing that. (Assuming things are properly implemented of course). Because GOGrepo generally does a lot of file replacement and downloads 4 files simultaneously a lot , it's a nightmare for file fragmentation on Windows and any of the file systems it uses (so Linux should still pre-allocate on Windows filesystems).

I implemented this because I was migrating my GOG collection between two external USB HDDs and due to the fragmentation it was going to take over a week. It also made verification take way longer but I didn't realise that at the time. Fragmentation has been vastly less of a problem since this was implemented.
Posted October 24, 2022
A small issue, Kalanyr.

I noticed that if one downloads a game, verifies it, then deletes the files, downloads them again (without any update), and reruns verify --- the newly downloaded files are skipped, rather than verified, presumably because they are marked as pre-verified. I think downloading a file should clear the pre-verify flag.
Posted October 24, 2022
avatar
mrkgnao: A small issue, Kalanyr.

I noticed that if one downloads a game, verifies it, then deletes the files, downloads them again (without any update), and reruns verify --- the newly downloaded files are skipped, rather than verified, presumably because they are marked as pre-verified. I think downloading a file should clear the pre-verify flag.
Hmmmm. That's an interesting one. Download does not change the manifest. Maybe the best answer here is to store a bit of meta data about the verified file, like the file size and last modified date when it's verified. That way reverifications happen if things that could potentially cause data loss like copying the file occur, and pre-verification could time out, allowing files to be revalidated every so often.
Posted October 25, 2022
avatar
mrkgnao: A small issue, Kalanyr.

I noticed that if one downloads a game, verifies it, then deletes the files, downloads them again (without any update), and reruns verify --- the newly downloaded files are skipped, rather than verified, presumably because they are marked as pre-verified. I think downloading a file should clear the pre-verify flag.
From time to time, I refresh my manifest.

python3 gogrepoc.py update -strictverify -lang en -os windows mac -skiphidden -full -resumemode noresume
Posted October 25, 2022
avatar
mrkgnao: A small issue, Kalanyr.

I noticed that if one downloads a game, verifies it, then deletes the files, downloads them again (without any update), and reruns verify --- the newly downloaded files are skipped, rather than verified, presumably because they are marked as pre-verified. I think downloading a file should clear the pre-verify flag.
avatar
MatrixRaven: From time to time, I refresh my manifest.

python3 gogrepoc.py update -strictverify -lang en -os windows mac -skiphidden -full -resumemode noresume
Of course. So do I.
Posted October 25, 2022
avatar
MatrixRaven: From time to time, I refresh my manifest.

python3 gogrepoc.py update -strictverify -lang en -os windows mac -skiphidden -full -resumemode noresume
avatar
mrkgnao: Of course. So do I.
That fixes the problem...
Posted October 28, 2022
avatar
mrkgnao: I have added ymgve's "extra slashes" solution (see here) to my copy of gogrepoc. It solved all my current MD5 XML issues.

I didn't try to add it to the GitHub repository because:
(a) it's a hack
(b) when I retired, I swore I would never use git ever again

So, if you want it, you can add it to your own local copy.

To do so:
1) Search for the following line:
tmp_md5_url = append_xml_extension_to_url_path(response.url)
2) Add the following two lines after it (with the same indentation!):
extra_slashes = '/' * (datetime.datetime.today().minute + 2) ### RS: Setting replacement string based on current minute 09/06/22
tmp_md5_url = extra_slashes.join(tmp_md5_url.rsplit('/',1)) ### RS: Adding extra final slashes to bypass GOG's XML issues 09/06/22
3) Save

What it does is add extra slashes based on the current minute (i.e. between 1 and 60 extra slashes), so if you you have an XML failure on a specific game, wait a minute and try just that game again.

I had 6 games with outstanding XML failures (most had been failing for weeks). I ran gogrepoc on these 6 games with the fix and 5 were resolved. I ran it on the 6th game again a minute or two later and it was resolved too.

This handles failures during update. There could also be failures during download, but I have very rarely seen any. If you have those, you can use a similar solution there too. I haven't tried it on real life failures, but I did add it to my code nonetheless and it doesn't seem to hurt anything.

To do that (optional):
1) Search for the following line:
chunk_url = append_xml_extension_to_url_path(response.url)
2) Add the following two lines after it (with the same indentation!):
extra_slashes = '/' * (datetime.datetime.today().minute + 2) ### RS: Setting replacement string based on current minute 09/06/22
chunk_url = extra_slashes.join(chunk_url.rsplit('/',1)) ### RS: Adding extra final slashes to bypass GOG's XML issues 09/06/22
3) Save

Thank you, ymgve.

P.S. I would appreciate a code review, if someone is up to it.
Hello mrkgnao, is your code still needed to be inserted into the newest september build, to avoid .xml errors?
I have run the gogrepoc today after half year, and I am getting some errors :(

openttd will be an example of problematic game for me.
or ultima_underworld_1
Post edited October 28, 2022 by MMLN
Posted October 28, 2022
logs for beautiful_isolation_game


18:18:39 | loading local manifest...
18:18:39 | loading token...
18:18:39 | loading local resume manifest...
18:18:39 | fetching game product data (page 1)...
18:18:41 | scanning found "beautiful_desolation_game" in product data!
18:18:41 | saving resume manifest...
18:18:41 | saved resume manifest
18:18:41 | (1 / 1) fetching game details for beautiful_desolation_game...
18:19:27 | request failed: HTTPSConnectionPool(host='gog-cdn-lumen.secure2.footprint.net', port=443): Read timed out. (3 retries left) -- will retry in 5s...
18:20:13 | request failed: HTTPSConnectionPool(host='gog-cdn-lumen.secure2.footprint.net', port=443): Read timed out. (2 retries left) -- will retry in 5s...
18:20:58 | request failed: HTTPSConnectionPool(host='gog-cdn-lumen.secure2.footprint.net', port=443): Read timed out. (1 retries left) -- will retry in 5s...
18:21:38 | unexpected connection error fetching md5 data for setup_beautiful_desolation_1.0.7.3_win_(64bit)_(56745)-1.bin This error may be temporary. Please retry in 24 hours.
18:21:38 | The handled exception was:
18:21:38 |
Traceback (most recent call last):
File "C:\Program Files\Python39\lib\site-packages\urllib3\response.py", line 438, in _error_catcher
yield
File "C:\Program Files\Python39\lib\site-packages\urllib3\response.py", line 519, in read
data = self._fp.read(amt) if not fp_closed else b""
File "C:\Program Files\Python39\lib\http\client.py", line 455, in read
n = self.readinto(b)
File "C:\Program Files\Python39\lib\http\client.py", line 499, in readinto
n = self.fp.readinto(b)
File "C:\Program Files\Python39\lib\socket.py", line 704, in readinto
return self._sock.recv_into(b)
File "C:\Program Files\Python39\lib\ssl.py", line 1241, in recv_into
return self.read(nbytes, buffer)
File "C:\Program Files\Python39\lib\ssl.py", line 1099, in read
return self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Program Files\Python39\lib\site-packages\requests\models.py", line 753, in generate
for chunk in self.raw.stream(chunk_size, decode_content=True):
File "C:\Program Files\Python39\lib\site-packages\urllib3\response.py", line 576, in stream
data = self.read(amt=amt, decode_content=decode_content)
File "C:\Program Files\Python39\lib\site-packages\urllib3\response.py", line 541, in read
raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
File "C:\Program Files\Python39\lib\contextlib.py", line 135, in __exit__
self.gen.throw(type, value, traceback)
File "C:\Program Files\Python39\lib\site-packages\urllib3\response.py", line 443, in _error_catcher
raise ReadTimeoutError(self._pool, None, "Read timed out.")
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='gog-cdn-lumen.secure2.footprint.net', port=443): Read timed out.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:\Backup\GOG\gogrepoc.py", line 242, in request
response = session.get(url, params=args,stream=stream,timeout=HTTP_TIMEOUT)
File "C:\Program Files\Python39\lib\site-packages\requests\sessions.py", line 555, in get
return self.request('GET', url, **kwargs)
File "C:\Program Files\Python39\lib\site-packages\requests\sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "C:\Program Files\Python39\lib\site-packages\requests\sessions.py", line 697, in send
r.content
File "C:\Program Files\Python39\lib\site-packages\requests\models.py", line 831, in content
self._content = b''.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b''
File "C:\Program Files\Python39\lib\site-packages\requests\models.py", line 760, in generate
raise ConnectionError(e)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='gog-cdn-lumen.secure2.footprint.net', port=443): Read timed out.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Program Files\Python39\lib\site-packages\urllib3\response.py", line 438, in _error_catcher
yield
File "C:\Program Files\Python39\lib\site-packages\urllib3\response.py", line 519, in read
data = self._fp.read(amt) if not fp_closed else b""
File "C:\Program Files\Python39\lib\http\client.py", line 455, in read
n = self.readinto(b)
File "C:\Program Files\Python39\lib\http\client.py", line 499, in readinto
n = self.fp.readinto(b)
File "C:\Program Files\Python39\lib\socket.py", line 704, in readinto
return self._sock.recv_into(b)
File "C:\Program Files\Python39\lib\ssl.py", line 1241, in recv_into
return self.read(nbytes, buffer)
File "C:\Program Files\Python39\lib\ssl.py", line 1099, in read
return self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out

>>>The error continues, but the forum SW does not allow me to post more...
Post edited October 28, 2022 by MMLN
Posted October 28, 2022
looks like I can continue:

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Program Files\Python39\lib\site-packages\requests\models.py", line 753, in generate
for chunk in self.raw.stream(chunk_size, decode_content=True):
File "C:\Program Files\Python39\lib\site-packages\urllib3\response.py", line 576, in stream
data = self.read(amt=amt, decode_content=decode_content)
File "C:\Program Files\Python39\lib\site-packages\urllib3\response.py", line 541, in read
raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
File "C:\Program Files\Python39\lib\contextlib.py", line 135, in __exit__
self.gen.throw(type, value, traceback)
File "C:\Program Files\Python39\lib\site-packages\urllib3\response.py", line 443, in _error_catcher
raise ReadTimeoutError(self._pool, None, "Read timed out.")
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='gog-cdn-lumen.secure2.footprint.net', port=443): Read timed out.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:\Backup\GOG\gogrepoc.py", line 242, in request
response = session.get(url, params=args,stream=stream,timeout=HTTP_TIMEOUT)
File "C:\Program Files\Python39\lib\site-packages\requests\sessions.py", line 555, in get
return self.request('GET', url, **kwargs)
File "C:\Program Files\Python39\lib\site-packages\requests\sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "C:\Program Files\Python39\lib\site-packages\requests\sessions.py", line 697, in send
r.content
File "C:\Program Files\Python39\lib\site-packages\requests\models.py", line 831, in content
self._content = b''.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b''
File "C:\Program Files\Python39\lib\site-packages\requests\models.py", line 760, in generate
raise ConnectionError(e)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='gog-cdn-lumen.secure2.footprint.net', port=443): Read timed out.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Program Files\Python39\lib\site-packages\urllib3\response.py", line 438, in _error_catcher
yield
File "C:\Program Files\Python39\lib\site-packages\urllib3\response.py", line 519, in read
data = self._fp.read(amt) if not fp_closed else b""
File "C:\Program Files\Python39\lib\http\client.py", line 455, in read
n = self.readinto(b)
File "C:\Program Files\Python39\lib\http\client.py", line 499, in readinto
n = self.fp.readinto(b)
File "C:\Program Files\Python39\lib\socket.py", line 704, in readinto
return self._sock.recv_into(b)
File "C:\Program Files\Python39\lib\ssl.py", line 1241, in recv_into
return self.read(nbytes, buffer)
File "C:\Program Files\Python39\lib\ssl.py", line 1099, in read
return self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Program Files\Python39\lib\site-packages\requests\models.py", line 753, in generate
for chunk in self.raw.stream(chunk_size, decode_content=True):
File "C:\Program Files\Python39\lib\site-packages\urllib3\response.py", line 576, in stream
data = self.read(amt=amt, decode_content=decode_content)
File "C:\Program Files\Python39\lib\site-packages\urllib3\response.py", line 541, in read
raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
File "C:\Program Files\Python39\lib\contextlib.py", line 135, in __exit__
self.gen.throw(type, value, traceback)
File "C:\Program Files\Python39\lib\site-packages\urllib3\response.py", line 443, in _error_catcher
raise ReadTimeoutError(self._pool, None, "Read timed out.")
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='gog-cdn-lumen.secure2.footprint.net', port=443): Read timed out.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:\Backup\GOG\gogrepoc.py", line 242, in request
response = session.get(url, params=args,stream=stream,timeout=HTTP_TIMEOUT)
File "C:\Program Files\Python39\lib\site-packages\requests\sessions.py", line 555, in get
return self.request('GET', url, **kwargs)
File "C:\Program Files\Python39\lib\site-packages\requests\sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "C:\Program Files\Python39\lib\site-packages\requests\sessions.py", line 697, in send
r.content
File "C:\Program Files\Python39\lib\site-packages\requests\models.py", line 831, in content
self._content = b''.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b''
File "C:\Program Files\Python39\lib\site-packages\requests\models.py", line 760, in generate
raise ConnectionError(e)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='gog-cdn-lumen.secure2.footprint.net', port=443): Read timed out.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Program Files\Python39\lib\site-packages\urllib3\response.py", line 438, in _error_catcher
yield
File "C:\Program Files\Python39\lib\site-packages\urllib3\response.py", line 519, in read
data = self._fp.read(amt) if not fp_closed else b""
File "C:\Program Files\Python39\lib\http\client.py", line 455, in read
n = self.readinto(b)
File "C:\Program Files\Python39\lib\http\client.py", line 499, in readinto
n = self.fp.readinto(b)
File "C:\Program Files\Python39\lib\socket.py", line 704, in readinto
return self._sock.recv_into(b)
File "C:\Program Files\Python39\lib\ssl.py", line 1241, in recv_into
return self.read(nbytes, buffer)
File "C:\Program Files\Python39\lib\ssl.py", line 1099, in read
return self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Program Files\Python39\lib\site-packages\requests\models.py", line 753, in generate
for chunk in self.raw.stream(chunk_size, decode_content=True):
File "C:\Program Files\Python39\lib\site-packages\urllib3\response.py", line 576, in stream
data = self.read(amt=amt, decode_content=decode_content)
File "C:\Program Files\Python39\lib\site-packages\urllib3\response.py", line 541, in read
raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
File "C:\Program Files\Python39\lib\contextlib.py", line 135, in __exit__
self.gen.throw(type, value, traceback)
File "C:\Program Files\Python39\lib\site-packages\urllib3\response.py", line 443, in _error_catcher
raise ReadTimeoutError(self._pool, None, "Read timed out.")
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='gog-cdn-lumen.secure2.footprint.net', port=443): Read timed out.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:\Backup\GOG\gogrepoc.py", line 711, in fetch_file_info
md5_response = request(updateSession,tmp_md5_url)
File "D:\Backup\GOG\gogrepoc.py", line 256, in request
return request(session=session,url=url, args=args, byte_range=byte_range, retries=retries-1, delay=HTTP_RETRY_DELAY)
File "D:\Backup\GOG\gogrepoc.py", line 256, in request
return request(session=session,url=url, args=args, byte_range=byte_range, retries=retries-1, delay=HTTP_RETRY_DELAY)
File "D:\Backup\GOG\gogrepoc.py", line 256, in request
return request(session=session,url=url, args=args, byte_range=byte_range, retries=retries-1, delay=HTTP_RETRY_DELAY)
File "D:\Backup\GOG\gogrepoc.py", line 242, in request
response = session.get(url, params=args,stream=stream,timeout=HTTP_TIMEOUT)
File "C:\Program Files\Python39\lib\site-packages\requests\sessions.py", line 555, in get
return self.request('GET', url, **kwargs)
File "C:\Program Files\Python39\lib\site-packages\requests\sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "C:\Program Files\Python39\lib\site-packages\requests\sessions.py", line 697, in send
r.content
File "C:\Program Files\Python39\lib\site-packages\requests\models.py", line 831, in content
self._content = b''.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b''
File "C:\Program Files\Python39\lib\site-packages\requests\models.py", line 760, in generate
raise ConnectionError(e)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='gog-cdn-lumen.secure2.footprint.net', port=443): Read timed out.
18:21:38 | End exception report.
18:21:42 | saving manifest...
18:21:42 | saved manifest
18:21:42 | saving resume manifest...
18:21:42 | saved resume manifest
18:21:42 | --
18:21:42 | total time: 0:03:03.508046
18:21:42 | exiting...
Post edited October 28, 2022 by MMLN
Pages:
1
50
100
150
200
222
223
224
225
226
260
This is my favourite topic
Community
General discussiongogrepo.py - Python script for regularly backing up your purchased GOG collection for full offline enjoyment(3896 posts)(3896 posts)
(3896 posts)