It seems that you're using an outdated browser. Some things may not work as they should (or don't work at all).
We suggest you upgrade newer and better browser like: Chrome, Firefox, Internet Explorer or Opera

×
avatar
osm: get stuff done vs raise the world's entropy is the real q here.
avatar
timppu: None of that really matters considering the climate change, identity politics and US elections.

Also, hungry children in Africa. Especially them.
uhh.. k den.

Though personally I'll stick to doing stuff rather than waste time NOT applying Occam's Razor. There are just better ways to waste time in life. Involving Linux as well.

PS not hungry enough it seems to stop the Black Continent from being the ever main contributor to world population growth
I've read through all of the new posts (admittedly skimming some things for lack of time available), and I'll make a full reply later today when I have the time, but here are a few things to keep in mind:

I don't have any SSD, and I use only HDD.

I realized that my USB stick is USB 2.0 (I think, but it doesn't say on it), so I'm going to get some 3.0 ones today and see if that works better.

My biggest problem is NOT the fact that GRUB keeps being corrupted and needs to be reinstalled. That's only a very minor annoyance by comparison. By biggest problem by far is the fact that once I fix the corrupted GRUB on the HD, the USB stick no longer boots, no matter what I do! So it becomes completely unusable, unless the GRUB on the HD gets corrupted again, at which point, I'm then able to boot from the GRUB on the USB stick to recover the GRUB on the HD, which then makes the GRUB and Linux on the USB stuck once again unbootable!

And my other huge problem is that I can't seem to access my HD or other USB ports once I'm running Linux on a USB stick. As for the slowness, I think and hope that will be solved by using USB 3.0 sticks.
avatar
HeresMyAccount: By biggest problem by far is the fact that once I fix the corrupted GRUB on the HD, the USB stick no longer boots, no matter what I do!
Ok so that's your firmware.. in short, find the manual, there's nothing you can do about it in Linux.
And my other huge problem is that I can't seem to access my HD or other USB ports once I'm running Linux on a USB stick.
Understood. Let's wait till you have time to diagnose it further.
avatar
HeresMyAccount: And my other huge problem is that I can't seem to access my HD or other USB ports once I'm running Linux on a USB stick.
Ensure the HDD is mounted. To do it on GUI open start menu -> acessories -> disks
Alright, so now I have time to reply to everyone:

avatar
clarry: OK, I've never used Mint and I didn't know it has a broken installer. GRUB can be installed manually once you get a Linux system up and running. I think you're after a BIOS/GPT setup, with an MBR that has some partition marked bootable. Idk if there's a step-by-step walkthrough, and I'm not really going to look for one, but these are the instructions I'd start with:

[url=https://wiki.archlinux.org/index.php/GRUB#GUID_Partition_Table_(GPT)_specific_instructions]https://wiki.archlinux.org/index.php/GRUB#GUID_Partition_Table_(GPT)_specific_instructions[/url]
https://www.gnu.org/software/grub/manual/grub/html_node/BIOS-installation.html#BIOS-installation
https://wiki.archlinux.org/index.php/GRUB#Intel_BIOS_not_booting_GPT
I'll check those, thanks.

avatar
clarry: It's the hardware, primarily. The problem is that flash can't just write a byte onto the media. Instead, if I change one byte in a file, the flash will have to erase an entire block (multiple kilobytes, or worse..) and then rewrite all of it, with the new changes included. Erasing also tends to be really slow, especially on flash optimized for density. (Of course, writes are also slow on cheap flash, but they're nowhere near as bad as erases). Of course cheap flash doesn't include much cache to speed this up either.
There's a cache on these things? Why would erasing a block be slower than writing data to fill it? Isn't erasing the same as just writing 0 everywhere, and if so then why doesn't it do that instead if it's quicker?

avatar
clarry: I suppose a filesystem could be tuned for flash, making sure it uses these multi-kilobyte chunks as the minimum size of allocation. That wastes some space. Unfortunately you'll still need to start erasing, eventually.
Well I guess when I format it I could say to use the smallest block/sector/whatever size available, though I don't remember it giving me the option in the Mint installer or even in GParted. When you say I'll have to erase eventually, what do you mean? Are you referring to the fact that it's "erasing" whenever it writes a file, like you said above, or do you mean because I'll have to manually erase files at some point? That doesn't really worry me, because I plan to install and configure everything how I want and then make it read-only after that, so there won't be any erasing in the long run. I just need it quick enough that I'll be able to set everything up initially.

avatar
clarry: It's really hard to say, there are so many possibilities. The first one I'd check though is the size. People can correct me if I'm wrong but I got the impression that Mint is pretty large, whereas I'd imagine Porteus is optimized for size, given its intended use.
Mint, once installed seems to be about 6.5 GB, though the installer is about 1.8 GB. Porteus is only about 300 MB for the installer but I'm not sure of the size after it's installed. In any case, it only took me a few seconds to install Porteus (though I couldn't boot from it, but I'm not sure whether that's because of an error in the installation or just a difficulty booting from USB drives in general), whereas Mint took hours to install the 6.5 GB, so it's still not a comparably similar speed.

avatar
clarry: Different chips, possibly. Just because a stick looks the same doesn't mean the hardware inside is the same. If I had such sticks and wanted to understand the difference, I'd first of all do controlled tests by raw writes into the block device (i.e. dd onto /dev/sda or whatever the device is) and try different block sizes starting at 512 bytes, going up to maybe 32k or even 64k. That would give me an idea about what the hardware is capable of when you take filesystem and other things out of the equation.
Yeah, I guess I could try something like that, but I'll see how the USB 3.0 ones work before I bother with that, since it may become a moot point. I should be able to get 3.0 this afternoon.

avatar
clarry: Unlikely. I'd expect disk I/O to be painfully slow, but this shouldn't affect things running from RAM and on the CPU. I suspect you're being slowed down by the vesa framebuffer (that's what you get for graphics if you're not using UEFI and have no other working graphics drivers.. yeah you mentioned xforcevesa).
I don't see how there's any way it could possibly be because of the vesa or anything related to the GeForce. Like I said before, I'm using Linux on my HD right at this very moment, and it's set up with those same parameters, and it runs great! Also, when I had experimentally tried to install Linux on a USB the previous time (which actually was 3.0 as it turns out), though I was never able to get it to boot properly after the initial time and in exceptional cases (when the GRUB on the HD was corrupted), I had the video drivers and settings the same way then as well, and I wasn't noticing a slow-down at all. So how could that be the cause?

avatar
clarry: However disks should show up in dmesg when the kernel probes them. Look for sd (scsi) or hd (ide). If they do not show up, something up the chain is failing. Linux dmesg is rather messy (sigh, why can't they do it like OpenBSD) but you should be able to follow the chain which depends on your hardware. For example (in case of SATA) from pci to ahci to ata to scsi to sd. Let's take a look at the dmesg on my desktop (AMD X370 chipset), I have two sata controllers and two SSDs (cutting some excess fluff):...
Well that makes sense, but I'm not sure I follow the lines that you printed after it. They looked like a bunch of nearly identical lines except for a slightly different number after it said "SATA", and I don't know where you got the idea that 2 and 4 are the primary ones, nor am I entirely sure what you even mean by that in this context (I don't know a whole lot about hardware, to be honest, because I'm a programmer and I deal mostly with pure software).

In any case, when I type dmesg I get a HUGE and incredibly long list of all kinds of different stuff (like maybe over 100 pages!).

avatar
clarry: So I have found my disks, and I can also tell that the first one has four partitions (sda1-sda4) and the second one has one partition (sdb1). Those are things I see under /dev, for example /dev/sda is the first drive and /dev/sda1 is its first partition. Assuming it is formatted and Linux knows the filesystem, I should be able to just say mount /dev/sda1 /mnt and it will be mounted under /mnt, or there will be some error that gives me a clue about what's going wrong.
If I remember correctly, I think that when I looked in the dev folder I didn't find any disks, nor when when to the directory represented by the computer icon (I don't think its the root, but something similar - I think they may just call it "~", but I'm not sure). In any case, I doubt it was mounted, because usually it appears on the left side whether it's mounted or not, and then I can mount it if I want, but this time it wasn't even there, so it was missing to an extent even greater than if it was just unmounted.

avatar
clarry: What if those dev entries are missing? Then you would need to know a bit about your hardware and follow up the chain in dmesg and see who's the missing link. After that, it's more hardware specific, you need to figure out why the driver isn't probing or attaching (it could be something as simple as a driver missing from the kernel, it could be some kind of error, could be firmware acting up or misconfigured bios (devices disabled?), it could be something that requires an expensive oscilloscope to diagnose!). But knowing which part is the failing link helps one get started looking for a fix.
Well here's what doesn't make sense to me about this theory. First of all, why would something be missing from the kernel if I used the exact same installer that I used every time I've installed Linux before, including the one that I'm using right now, and I don't have a problem at all on this one? And for that matter, how could there be a hardware problem (like with the hard drive, or for that matter, every USB stick that I plug in), for the same reason, considering that all of those things work when I'm booted into Mint on my HD? I'm not trying to argue, but I just want to understand your reasoning because it seems to contradict the evidence, though I may be overlooking some different circumstance or something.

And I don't see how I can possibly get my hands on an oscilloscope, let alone figure out how to use it to do what I need to do here, but I doubt that would be necessary, anyway.

avatar
clarry: Ok yeah so I guess you're talking about Mint's broken installer, not GRUB's own installer, right? GRUB's own tools really should do exactly what you tell it to do and not mess with anything else (but of course, as you know, bugs are not unheard of.. just somewhat unlikely in case of GRUB).
Right, I was referring to the fact that every single time I install Mint onto a USB drive, it always automatically corrupts the GRUB which is already on the HD, and then I have to manually reinstall GRUB directly, in order to fix it. This happens regardless of whether I specify within the installer to place GRUB on the HD or on the USB.

To be continued...
Post edited October 30, 2020 by HeresMyAccount
avatar
clarry: It's the hardware, primarily. The problem is that flash can't just write a byte onto the media. Instead, if I change one byte in a file, the flash will have to erase an entire block (multiple kilobytes, or worse..) and then rewrite all of it, with the new changes included. Erasing also tends to be really slow, especially on flash optimized for density. (Of course, writes are also slow on cheap flash, but they're nowhere near as bad as erases). Of course cheap flash doesn't include much cache to speed this up either.
avatar
HeresMyAccount: There's a cache on these things? Why would erasing a block be slower than writing data to fill it? Isn't erasing the same as just writing 0 everywhere, and if so then why doesn't it do that instead if it's quicker?
My understanding is that the way flash memory works is something like this (someone correct me if I'm wrong):
* You can't write to the memory if you don't erase it first (either that, or you can change bits in only one direction).
* Flash memory, when erased, actually has a bit pattern of all 1's, not all 0's, so writing all 0's to it would actually be the opposite of erasing it.

Edit: Just checked Wikipedia, and it turns out that, indeed, 1's can be changed to 0's without erasing the whole block, but doing the reverse isn't possible without erasing it.
Post edited October 30, 2020 by dtgreene
However, I've read that there's an option to run it with -b I think (I wrote it down so I'd double check if I was planning to try it), and that will install Linux without placing GRUB anywhere! That might possibly prevent the corruption problem, but here's a VERY IMPORTANT question, because it might possibly solve my problem: if I were to install Linux onto a USB drive without any GRUB at all, would that then hypothetically:

- be able to boot in either BIOS or UEFI mode for any computer that can boot from USB

- be limited to one particular mode

- never be able to boot at all

- be able to boot only in a computer that already has GRUB installed on its HD

Which of these do you think is accurate? My theory is that it might possibly be the first option but couldn't be the second option, because if there's no extra partition with the BIOS or UEFI data then it's not specifying which one to use, so it should either work in both or neither, since it's not being preferential in any way. It might possibly be the third option, but on the other hand, a lot of people have only one OS installed on their computer and it boots just fine without any boot loader or other extra stuff at all (as far as I can tell), so it seems like it should boot just because there'd be nothing to get in the way, referring back to the first option - does that make sense? Or the fourth option might be the case, because if Linux requires GRUB (or another boot loader) to boot, and if GRUB is on the HD already then it should be possible to recognize and boot the USB drive that way, even if the USB drive doesn't have GRUB, but then if it's placed into a computer without GRUB, it may not recognize the USB to boot - but on the other hand it might simply because Linux is on it, so it's back to option 1.

HOWEVER, keep in mind that as much as I always have problems booting from USB drives, I can ALWAYS boot from the installer drive and run live mode! Does the installer have a boot loader? It seems to have GRUB, or some version of it, but the options on it are very different (like Compatibility Mode). So I really don't know why that always seems to work but the ones that I create just don't.
And to reply to your next post:

avatar
clarry: By the way -- and I'm sorry if we talked about this already and I forgot or missed it -- but you really should check your computer's (or motherboard's) manual and see if there's a way to select the boot device. Because I've never seen one that doesn't allow it. Not saying there couldn't be such a thing.. also in addition to setting boot order in BIOS or UEFI, many PCs additionally have some alternate key (often something like F12 or F10 or F1 or whatever) that you can press during POST and it'll bring up a boot menu from which you can directly select the boot device, without going through the UEFI jank. This too should be documented in the manual.
I'll look into that also, thanks for the tip. I hope I can just use a boot menu, because that would probably be the most convenient way.

avatar
clarry: Ok, so they went the way of the BIOS GPT installation described in my links above. And your problem is still that the Mint installer messes up the bootloader on your HD? Then I'd try to install GRUB manually instead of letting Mint botch it up. Or just unplug the HD for the duration of the installation, there's no way it'll mess with a drive sitting in your desk drawer ;-)
Well, like I said, that's the least of my worries, because it won't help me boot from the USB, and I don't see how I can prevent Mint from botching it up unless I use the option to disable it from installing GRUB (which may possibly still botch it up anyway). I don't see how I could unplug the HD, considering that I need to reboot directly from Windows to get to the installer, so I'd have to open up and unscrew the HD while the computer's still running! Though if I can get the boot menu to work then that would be an option, but honestly it's less trouble to just reinstall GRUB after it's corrupted.

avatar
clarry: There's generally no need for padding between partitions, as long as the partitions are aligned and their size is some multiple of the alignment size (which depends on a lot of things, including the sector size and hardware blocks on a flash drive which I talked about earlier). The first partition is preceded by the MBR, GPT, plus any space that might be reserved for a legacy bootloader, disklabel, or whatever else one might put there. So you can't start the first partition at the very first byte of the HD. Thus the starting offset of the first partition is rounded (aligned) somewhat arbitrarily to 1 megabyte. The alignment could be smaller than that, but it doesn't hurt to sacrifice one MB given the size of current drives.
Oh, so that was supposed to be that way after all? That's good to know, because I had considered trying it the next time without the 1 MB padding, thinking that it may work better that way, but I guess not. So the MBR, GPT and legacy boot loader all fit into 1 MB? Actually, the legacy boot loader I guess is the 2 MB BIOS partition that I created as the first one, but the other stuff is within the first MB which was offset as padding? I'm just trying to understand all of the details, because hopefully that will enable me to do this correctly.

avatar
clarry: When you're in UEFI mode, you get a different display driver than when you're in BIOS mode. The vesa framebuffer in BIOS mode can be ridiculously slow. UEFI framebuffer isn't guaranteed to be fast either but in my experience it always is much faster than vesa fb.
Well even when I was running in BIOS mode I don't think that the display or card had anything to do with the slow-down, because the mouse cursor could still move around fluidly, and when I did certain things, like move windows around, they happened pretty much instantly, with visual results, but most things were just slow.
avatar
Dark_art_: Just like Hard drives and sata SSD's can have roughly the same speed reading or writing large files, when it comes to lots of small files, SSD's are a order of magnitude faster.
High cache pen drives helps but tend to be expensive..

There's a "new" rating system for memory cards to deal with this issue, A1 and A2 rating. Not sure if is aplicable to pen drives.
Older ratings deal with maximum transfer speed but smatphones with apps installed on the card were notoriously slow in some cases. That's why the A1/A2 rating was implemented.
I have a couple of Sandisk A1 rated memory cards and they are quicker to copy small files compared to other cards I have (tbf, the other cards are older). Can even feel the diference using it on raspberry's GUI.
I'm not sure where to even find the rating. I'm lucky I could even find the USB version of these things. I get them from a store that has huge troughs of them, not in a box or any packaging. Each trough has ones of a different color, and each color represents a different size and USB version number. They're very cheap, but like I said, historically I've always had very good results with the USB 3.0 ones, so I'll be really happy this afternoon when I get more of those.

avatar
Dark_art_: Porteus runs on the RAM (meaning it's way faster to use), is much smaller size and focus on using less resources. I also had issues with Porteus but was quite some time ago.

If your pen drives are "made" by your local shop, the most likely is they buy very cheap chinese drives and print a logo on them. If this is the case they are usually very slow...

Depending on a lot of factors, bigger drives are usually faster.

USB 3.0 (now called 3.1 gen1) connectors are blue (sometimes red) and have more physical pins, 9 vs 4.,and both the pen and the computer need to support 3.0 standard.
USB3 is faster but cheaper drives still suffer from the small files problem,

The fastest device to install a OS is a USB SSD., 2.5" is quite bulky but some m.2 are pocket size, although way more expensive.
A A1 (or better A2) rated memory card with a USB3 card reader is the next best, preferably 64Gb or more.
I really only need small ones (even 8 GB would be fine) and they have to be cheap, because I plan to use a bunch of them to make numerous duplicates. They have "Made in Taiwan" written on them.
Post edited October 30, 2020 by HeresMyAccount
That was everything substantial up to that point. After that:

clarry, I'll have to look at the manual next to see what could possibly be wrong with my firmware, but this is a very new and great computer, so I don't see why there'd be anything wrong with it.

Dark_art_, thanks for the tip about checking the disks - I'll try that the next time I try to install it, if I get the same problem again.

dtgreene, then it could be written with all 1 instead of 0, but either way, if it's quicker than erasing then it should just do it that way. I still don't see why it's quicker though. As for changing 1 to 0 but not being able to do the reverse... that's weird. I guess it explains why 1 is considered the erased state, but still, they have these things set up oddly.
avatar
HeresMyAccount: Why would erasing a block be slower than writing data to fill it? Isn't erasing the same as just writing 0 everywhere, and if so then why doesn't it do that instead if it's quicker?
Flash is written in bits, but erased in large blocks.


I just check in on this thread occasionally, to see how the saga is unfolding, so forgive me if you've already answered this:

When you installed Mint to your USB stick, did you just do a normal install, like you'd do to a regular HDD/SDD?
That's not a good idea, as things like /var/log may put a strain (both in performance and wear and tear) on slow devices like cheap USB sticks.

Distros meant for USB sticks will do things like writing file system changes to RAM and only (optionally) persist them to the USB drive on shutdown.
Well I don't know how normal the install was, but I used the "something else" method to specify stuff. Lately I've been trying to use something referred to as a "hybrid" install, whereby I have 3 partitions - one for BIOS, one for UEFI and one for Linux Mint (to ensure maximum compatibility in all modes). I'm still having huge problems though, as I've explained above.

But concerning wear and tear, I tried using Porteus instead, and seemed to install it, but couldn't boot from it, though I don't know whether that's because of my booting problems in general or because it didn't install correctly or what. In any case, I'm using Mint but I plant to disable writing to the drive once everything is set the way I want it (it will never need to be reconfigured in any way after that, and any files saved will be to a different drive).

Also, I just got my USB 3.0 sticks so I'll be messing with that pretty soon!
I sometimes pop in to see how things are going, and try and learn something.

Right now it's a toss-up whether it's a never-ending story (love that movie), and/or an educational journey ;)

Ladies and gentlemen, state your bets! Will op ever manage to boot into Linux succesfully? Odds are 14-1 against according to the bookies.

Taking bets now.

*grabs pencil*
I wish that was funny, but it's sad.
Experiment and see if you can get Knoppix booting off those 8GB & 16GB USB Sticks you have.
Knoppix is a highly specialized linux distro designed to be booted off USB sticks/CDrom/Dvd media.
I actually have a Knoppix DVD that I got many years ago, but it didn't occur to me that I could also install it onto a USB drive. I'll try it! Are there any goofy problems that I should potentially be aware of? Can I set the same video card parameters in GRUB (nomodeset xforcevesa) if I have driver problems? Does it corrupt GRUB on the HD when I install Knoppix onto the USB?

EDIT: Oh, but I can install stuff into the main partition where Knoppix is installed, right? Because it's essential that I be able to do that, and change and save new configurations all on that partition.
Post edited October 30, 2020 by HeresMyAccount