It seems that you're using an outdated browser. Some things may not work as they should (or don't work at all).
We suggest you upgrade newer and better browser like: Chrome, Firefox, Internet Explorer or Opera

×
Community
General discussiongogrepo.py - Python script for regularly backing up your purchased GOG collection for full offline enjoyment(4006 posts)(4006 posts)
(4006 posts)
Pages:
1
50
99
100
101
102
103
150
200
250
268
This is my favourite topic
Posted May 16, 2018
avatar
Kalanyr: I very strongly recommend against removing pre-allocation on NTFS / FAT variant drives as the normal behaviour will lead to extreme file fragmention over time which is why it was introduced (if you must you should also cut simultaneous downloads to 1 and try to avoid other write operations on the drive at the same time, it's probably even wise to redirect the logging to another disk).
I'm using a 4 disk Synology NAS running in their Hybrid RAID configuration. As far as I'm aware, fragmentation with this setup is not an issue.
Posted May 16, 2018
avatar
Kalanyr: I very strongly recommend against removing pre-allocation on NTFS / FAT variant drives as the normal behaviour will lead to extreme file fragmention over time which is why it was introduced (if you must you should also cut simultaneous downloads to 1 and try to avoid other write operations on the drive at the same time, it's probably even wise to redirect the logging to another disk).
avatar
ikrananka: I'm using a 4 disk Synology NAS running in their Hybrid RAID configuration. As far as I'm aware, fragmentation with this setup is not an issue.
I'm surprised that a NAS of that complexity is running NTFS at all. But yeah NAS is a situation where being able to turn pre-allocation off may make sense. I'll probably also add a switch that tries pre-allocation but suppresses errors for Linux. Or see if I can extract some information about the write target to automate stuff (there's several linux file systems where fallocate will always fail or isn't useful).
Posted May 16, 2018
avatar
ikrananka: I'm using a 4 disk Synology NAS running in their Hybrid RAID configuration. As far as I'm aware, fragmentation with this setup is not an issue.
avatar
Kalanyr: I'm surprised that a NAS of that complexity is running NTFS at all. But yeah NAS is a situation where being able to turn pre-allocation off may make sense. I'll probably also add a switch that tries pre-allocation but suppresses errors for Linux. Or see if I can extract some information about the write target to automate stuff (there's several linux file systems where fallocate will always fail or isn't useful).
Its not running NTFS, its file system is ext4. I'm also running Windows 10 and not Linux.
Post edited May 16, 2018 by ikrananka
Posted May 17, 2018
With Kalanyr taking over the maintenance of this project, I propose that a new topic be started by Kalanyr, so the first post can be modified with updated information whenever it is convenient for Kalanyr. People shouldn't have to read a month's worth of messages to learn that the project has changed hands.
Posted May 17, 2018
avatar
badon: With Kalanyr taking over the maintenance of this project, I propose that a new topic be started by Kalanyr, so the first post can be modified with updated information whenever it is convenient for Kalanyr. People shouldn't have to read a month's worth of messages to learn that the project has changed hands.
Been suggested, will be done soon. He wants to get it to a certain point before he rewrites the Readme and makes a topic.
Posted May 17, 2018
Am I seeing this correctly?
In the README.md file, the description of the download command shows -skipextras twice:

download [-h] [-dryrun] [-skipextras] [-skipextras] ...
Posted May 17, 2018
avatar
Kalanyr: I'm surprised that a NAS of that complexity is running NTFS at all. But yeah NAS is a situation where being able to turn pre-allocation off may make sense. I'll probably also add a switch that tries pre-allocation but suppresses errors for Linux. Or see if I can extract some information about the write target to automate stuff (there's several linux file systems where fallocate will always fail or isn't useful).
avatar
ikrananka: Its not running NTFS, its file system is ext4. I'm also running Windows 10 and not Linux.
Yeah, that's definitely a good reason for me to try and do file system determination (are you okay with me messaging you to try out some stuff on this while I'm working on it ?) , I currently just use the OS as a proxy (which is fine on Linux but less fine on Windows).


(On a related note I'm looking for someone who uses a Mac to do testing too).
Posted May 17, 2018
avatar
Kalanyr: Yeah, that's definitely a good reason for me to try and do file system determination (are you okay with me messaging you to try out some stuff on this while I'm working on it ?)
Yep, no problem with messaging me. I'll do anything I can to help.
Posted May 19, 2018
I can't successfully import a cookies.txt file into gogrepo, which is really important for me. My password is long and has characters that are considered special by the bash shell, so I can't use the normal gogrepo login.

I exported the cookies.txt from a fresh start of Firefox where the only site I visited or logged in to was gog.com. I then copied the cookies.txt file directly next to gogrepoc.py. I did not use gogrepoc.py login; I went straight to gogrepoc.py update. But I just keep getting the message "failed to load product data (are you still logged in?)".

I've tried to troubleshoot this, but I'm mostly just scratching my head at this point. I'd really appreciate any help and would be happy to give any info that would be wanted/needed.

(Random thought just now. Does it make any difference what I do with the browser window after I've exported the cookies? I left it up on gog.com and still logged in.)

For what it's worth:
I'm using the dev branch of Kalanyr's script.
I'm using Python 3.4 on CentOS 7. All required modules were installed with pip.
I'm using this cookie exporter extension on Firefox 59.0.2. (I believe this is the same extension Kalanyr uses?)
addons.mozilla.org/en-US/firefox/addon/cookies-txt/
Posted May 19, 2018
avatar
JethCalark: I can't successfully import a cookies.txt file into gogrepo, which is really important for me. My password is long and has characters that are considered special by the bash shell, so I can't use the normal gogrepo login.

I exported the cookies.txt from a fresh start of Firefox where the only site I visited or logged in to was gog.com. I then copied the cookies.txt file directly next to gogrepoc.py. I did not use gogrepoc.py login; I went straight to gogrepoc.py update. But I just keep getting the message "failed to load product data (are you still logged in?)".

I've tried to troubleshoot this, but I'm mostly just scratching my head at this point. I'd really appreciate any help and would be happy to give any info that would be wanted/needed.

(Random thought just now. Does it make any difference what I do with the browser window after I've exported the cookies? I left it up on gog.com and still logged in.)

For what it's worth:
I'm using the dev branch of Kalanyr's script.
I'm using Python 3.4 on CentOS 7. All required modules were installed with pip.
I'm using this cookie exporter extension on Firefox 59.0.2. (I believe this is the same extension Kalanyr uses?)
addons.mozilla.org/en-US/firefox/addon/cookies-txt/
It sounds like you're doing everything correctly.

Could you check if the file cookies.txt.tmp was generated in the same folder and if it has the GOG related stuff in it, with some basic cleaning up ?
Posted May 19, 2018
It seems GOG was down for a few hours today and now that the website is back I still can't continue my gogrepo download.

Do I just have to wait or is there anything I can do here?
Post edited May 19, 2018 by Nix31
Posted May 20, 2018
avatar
Nix31: It seems GOG was down for a few hours today and now that the website is back I still can't continue my gogrepo download.

Do I just have to wait or is there anything I can do here?
I would need to see the error message(s) you're getting to even guess.
Posted May 21, 2018
avatar
Nix31: It seems GOG was down for a few hours today and now that the website is back I still can't continue my gogrepo download.

Do I just have to wait or is there anything I can do here?
avatar
Kalanyr: I would need to see the error message(s) you're getting to even guess.
Right, sorry. It fixed itself shortly aftewards, but in any case I've attached the error screenshots from the log (couldn't directly post it here).
Attachments:
screenshot--2018.png (86 Kb)
screenshot--2018.png (113 Kb)
screenshot--2018.png (116 Kb)
Posted May 21, 2018
avatar
Kalanyr: I would need to see the error message(s) you're getting to even guess.
avatar
Nix31: Right, sorry. It fixed itself shortly aftewards, but in any case I've attached the error screenshots from the log (couldn't directly post it here).
For future reference that's a GOG has stopped responding to us error (it's the exact same error I got when GOG first went down), it should theoretically fix itself when GOG comes back up, if it doesn't self reserve within a few hours of GOG coming back up (and being able to download from the GOG website manually) , then report it (since it's such a generic error it could easily have other causes but if there's a recent known issue that's a safe bet)
Posted June 03, 2018
Just as a heads-up I will be traveling internationally in August, so if anything breaks spectacularly in that month fixes will probably be slow.

At this time I'm planning on taking my travel laptop with me , so I will be able to dev but stuff like travel / internet access / preplanned stuff on various days may mean I can't respond particularly quickly.
Pages:
1
50
99
100
101
102
103
150
200
250
268
This is my favourite topic
Community
General discussiongogrepo.py - Python script for regularly backing up your purchased GOG collection for full offline enjoyment(4006 posts)(4006 posts)
(4006 posts)