Hardware question - does bootloader and internal memory sit on the same flash?

[mvl]

Has anyone disassembled a 720 or otherwise debugged the firmware and determined whether the bootloader and internal memory sit on the same flash chip?

Since flash ram has a max shelf life of about 100,000 writes, and auto-wearlevels its usage, I'm wondering if overusing the internal ram until it starts failing means that the bootloader flash fails at the same time.

If so, given the cheap cost of SD cards, it seems a safer thing to run everything on SD, to maximize the life of the bootloader ram.

 

[canderson]

Has anyone disassembled a 720 or otherwise debugged the firmware and determined whether the bootloader and internal memory sit on the same flash chip?

Since flash ram has a max shelf life of about 100,000 writes, and auto-wearlevels its usage, I'm wondering if overusing the internal ram until it starts failing means that the bootloader flash fails at the same time.

If so, given the cheap cost of SD cards, it seems a safer thing to run everything on SD, to maximize the life of the bootloader ram.

When I was inside my unit, I neglected to see how they had managed this.

I've seen the term "bootloader" tossed around here, and unless TomTom has done a very, very naughty thing in their design, the REAL bootloader is not rewritable to begin with. If it were, an interrupted write of that section of memory would create a brick! Since I haven't heard much on that front here (thankfully) I have to assume something else is going on -- that the "real" bootloader, the part that is just bright enough to deal with the USB port and flash memory system, isn't what we're talking about. Instead, I'm guessing that the low level hardware interface and the thing that unpacks the operating system and application and puts them into DRAM and jumps to the space in DRAM where this stuff resides is what we're calling a bootloader and is the thing about which we're seeing a version number.

With that preface ...

Neither the low level hardware interface thing I described or the bootloader need be fiddled at all with flash rewrites related to firmware or user data or anything else. Flash memory is erased and rewritten in blocks. The low level code can occupy segments/blocks of flash that are never touched (except to be read) even with reflashing of firmware, rewriting of user configuration data, etc. The blocks are selectively managed.

So whichever way they've done this, there is no reason to think that the NVRAM area that is occupied by a "boot loader", whatever its function, is ever being abused by excessive erase/write cycles. The autoleveling you are talking about need not be applied to areas of NVRAM that aren't part of the blocks normally fiddled as part of the flash file system where all of the other really changeable stuff resides, and shouldn't be.

 

[mvl]

I agree that sharing the flash would be terrible design, but some things got me concerned:

1) aj84's experience that internal and bootloader seem to die in unison - in fact I can't remember a post with a tomtom that had dead internal flash and still operated on SD
2) the slightly less than 2GB size of the internal flash
3) the raw power of bootloader upgrades - they create SDHC compatibility, they create quickgps capability, etc
4) tomtom's tendency to assemble hardware components from vendors

Bootloaders can't be user upgraded, without a version (and hopefully checksum) validation step by the prior bootloader, so that protection may have minimized the brick rate out there

If you ever disassemble another one, check it out and let us know. I'm really hoping you'll find a separate low capacity flash/NV chip somewhere on the board.

 

[canderson]

I agree that sharing the flash would be terrible design, but some things got me concerned:

Actually, it's not a terrible design for the reasons I mentioned (segmenting via blocks). It's not all that common these days to see separate NVRAM used for a boot loader on a device that also has a big flash for the file system or whatever else.

What you do NOT want to do is rewrite the bootloader as part of reflashing an application. That's the big no-no. It's VERY rare that a user can even rewrite a true bootloader under any circumstances. If done well, the lowest level part of that can't even be addressed by the operating system -- the part that can reload everything else. I was addressing your concern about "overusing" the NVRAM where the true bootloader resides. As I say, that block (blocks) of memory should never get erased and rewritten UNLESS a special routine exists to rewrite the bootloader itself. So none of the other activity should be bothering it. You can erase and reflash the rest of the blocks to your hearts content -- or until it croaks.

1) aj84's experience that internal and bootloader seem to die in unison - in fact I can't remember a post with a tomtom that had dead internal flash and still operated on SD

All of which (above) isn't to say that the flash might not die entirely for reasons of its own, or that the processor find itself unable to address that area of memory where it resides. If that happened, you'd be toast in precisely the way you describe. I'm just knocking down the idea of the 100K cycle thing being any kind of an issue for the bootloader.

Bootloaders can't be user upgraded, without a version (and hopefully checksum) validation step by the prior bootloader, so that protection may have minimized the brick rate out there

Properly done, there should NEVER be a reason to rewrite the "real" one -- but then, we are talking about what the bootloader really IS in this device, and we may have an issue of terminology here.

If you ever disassemble another one, check it out and let us know. I'm really hoping you'll find a separate low capacity flash/NV chip somewhere on the board.

I won't feel at all fussed if I don't. A perfectly safe (from accidental screwups) system can be created without a separate memory for that.

 

[pianoCM]

Bootloader and memory normally are two different devices.

 

[canderson]

Bootloader and memory normally are two different devices.

We're not talking about DRAM here, but I assume you're talking about NVRAM vs. bootloader. Separating the bootloader from everything else used to be the most common approach for safety reasons (been a while since I've seen a boot ROM, too!), but unless the real bootloader is blown in the "OTP" region of a processor or otherwise internal to the processor, I'm seeing that most devices these days do not use a separate non-volatile memory device for it. Have also seen NVRAM products where the bottom end of memory can be "fused" after programming such that it can never be rewritten.

MVL - whatever their other failings at TomTom, paranoia about EU emissions regs certainly isn't misplaced. All of the "good stuff" (read: anything that might be directly tied to an address or data bus) is under tin and would require some very serious and careful desoldering work to reveal. If anyone has a "brick" they want to volunteer, I'll pull off the EMI shield on it and let you know what I found, but my TT720, Suzie, is already telling me that she's feeling violated.

 

[mvl]

What you do NOT want to do is rewrite the bootloader as part of reflashing an application.

When I refer to bootloader, I'm referring to the thing called "boot" in the app screen that irreversibly moved from 5.5015 to 5.5120 in the 8.350 software upgrade. As I understand, "system" loads "ttsystem" which loads the app in the bin folder. On a 720, "system" is the bootloader, stored somewhere in non-user-accessible space. If you place a new version of "system" on the INTERNAL drive, the real "system" will inspect it for an upgrade version and self update itself in the non-user-accessible space if the INTERNAL version is newer. But if the INTERNAL drive has an older version, same version, or is simply missing, the real "system" happily executes and starts up the tomtom.

So help me understand then - how exactly does flash controller wear leveling work? My assumption was that if a block was written repeatedly via the software, the flash controller would swap it out with another area less frequently used.

So, frequent rewrites to the temporary itinerary files, or to the mapsettings.cfg file, would ultimately mean that the controller will swap out a block with a less worn section (like a sector from the map). My worry is that the controller, in its lowest-level chip-based logic, may swap the blocks used for the true "system" file with the itinerary files, just for wear leveling purposes. If they're separate chips, or somehow intelligently not wear-loaded, then I wouldn't be as concerned.

Does my worry make sense? I'm concerned as replacing my tomtom would cost me $150 for a refurb + $100 for the map subscription + $70 for the plus traffic. If a cheapo $20 SD card could extend its brickable life, it may be a good insurance policy.

 

[canderson]

When I refer to bootloader, I'm referring to the thing called "boot" in the app screen that irreversibly moved from 5.5015 to 5.5120 in the 8.350 software upgrade. As I understand, "system" loads "ttsystem" which loads the app in the bin folder. On a 720, "system" is the bootloader, stored somewhere in non-user-accessible space. If you place a new version of "system" on the INTERNAL drive, the real "system" will inspect it for an upgrade version and self update itself in the non-user-accessible space if the INTERNAL version is newer. But if the INTERNAL drive has an older version, same version, or is simply missing, the real "system" happily executes and starts up the tomtom.

So help me understand then - how exactly does flash controller wear leveling work? My assumption was that if a block was written repeatedly via the software, the flash controller would swap it out with another area less frequently used.

So, frequent rewrites to the temporary itinerary files, or to the mapsettings.cfg file, would ultimately mean that the controller will swap out a block with a less worn section (like a sector from the map). My worry is that the controller, in its lowest-level chip-based logic, may swap the blocks used for the true "system" file with the itinerary files, just for wear leveling purposes. If they're separate chips, or somehow intelligently not wear-loaded, then I wouldn't be as concerned.

Does my worry make sense? I'm concerned as replacing my tomtom would cost me $150 for a refurb + $100 for the map subscription + $70 for the plus traffic. If a cheapo $20 SD card could extend its brickable life, it may be a good insurance policy.

I've managed to "declarify" this whole thing a bunch as I read back through prior posts. There's more than one topic being discussed (three, actually) and by not keeping them clearly separated, they're getting muddled.

First, I understand what TT users are referring to when they talk about the version level of the "Boot" program. Whatever that thing is, it appears with its own version number right alongside the versions of the OS and application, and the real odd man out, the 'GPS' code version. I can't get over the idea that the latter is actually the rev of the device driver that Linux is using to interface to the GPS chipset. What I have yet to understand is what TomTom means when they refer to "Boot". My "boot" shows up as version 5.5120, but what actual part of the system is it, and which low level functions does it perform? I'm trying to look at this in terms of an embedded Linux based system.

The "boot" code on a device these days serves several functions and usually performs its 'normal' job in stages as the device comes alive on a power up. I've never been clear which functions are actually being managed by that code on a TomTom. Further, I continue to suspect that there is an underlying bit of code that is even "lower" in the grand scheme of things than the thing we all identify as "Boot" with a version number.

It would probably be best to start with what we can actually see -- the ttsystem file. AFAIK, and that's not a great deal more than enough to be dangerous, ttsystem is an image of the entire system that will be moved into DRAM for execution, INCLUDING A "BOOT LOADER" which represents the first section of the ttsystem file! How the devil is that possible, you may well ask? Or better yet -- if the thing that is identified with a version and called "Boot" is located at the head of ttsystem, what on earth do we call the thing that's causing the ttsystem image to get stuffed into DRAM in the first place? You will find, using a hex editor, that ttsystem does indeed start with "boot" code, and is even identified as such (TTBL = TomTom Boot Loader).

So you can see where I'm having problems dealing with the terminology. Normally, the bootloader is separate code but not necessarily residing on separate physical memory hardware, and is certainly not part of the image. The job of a REAL bootloader in an environment like this is, among other things, to create a ramdisk system on the fly, unpack (if necessary) and move the OS image and application data to DRAM, start the OS, and by some mechanism, run a command line that allows the application to be started. So for obvious reasons, I'm having a problem calling something outside of the ttsystem file a bootloader while TomTom is evidently calling something INSIDE the image a bootloader (or "Boot", anyway) and giving it a version number.

The thing I've been calling "bootloader" appears to be a flash BIOS of some sort, and sports no version number on the version page at all. It also isn't visible in the flash file system, not that there is any reason it ever would unless a safe copy were kept there for some reason. So for now, let's try this on for size:

BIOS
BOOTLOADER
OS
APPLICATION

All of this can reside safely in the same physical piece of NVRAM (flash memory, in this case) if things are managed correctly. However, I've come to understand that at least in the past, if not the present, that the BIOS may be reflashable and that there have been releases where a new BIOS has been provided with a reflashing tool in the Linux kernel area. That makes me really, really nervous. A hard core design will assure that no matter what happens, the piece of the puzzle that functions to download a new and clean OS and application image to NVRAM cannot itself be overwritten by any user operation. That does mean you have to plan well ahead in your design, and this piece must be generic enough that you never design yourself into a box where you wish you could change it.

Ideally, a couple of things will be true that protect a design from serious error, especially by a user who is permitted to flash new code onto a consumer device...

First, the BIOS and/or Bootloader - whatever is the lowest level code - aren't even in address space that can be accessed by the operating system and application. This helps to assure that as the OS manages the flash memory tools, it has NO access to memory occupied by these critical bits of code. If the OS and application could access this space, it would be much easier for a corrupted OS or application to brick the unit such that it wasn't possible to bring in a clean version of whatever code was breaking things to begin with.

Second, as noted above, the lowest level code required for the device to bring in new OS and application code should itself be impossible to overwrite, even by intent. There are ways to avoid this entirely. One is to write this code into an OTP (one-time-programmable) area of memory in the processor itself. I'm currently doing that very thing with a Hitachi/Renesas chip design. Other mechanisms include post-programming fusing of the protected area of an external device to turn it into read-only memory. The old way was to use a separate piece of ROM memory to do it.

-----------------------
Back to the original issue -- there is no reason why such a region of memory that contained the critical low level code/BIOS/whatever would ever be touched once that code is loaded and in place, even if there is some sort of "auto leveling" function employed, which there may not be anyway.

Auto-leveling isn't a function of flash memory -- it's a function of the processor device that controls flash memory. Flash is just a bunch of bits like any other memory, without any real intelligence. A flash memory based device that includes a leveling function has it because there is a flash controller incorporated into the device along with the flash memory itself. That control can either be done directly by the device's primary processor or by a separate bit of hardware.

So just because a device like a TomTom incorporates flash memory, this does not mean that either 1) there is a leveling mechanism (either via hardware, or via software from the file system manager of the OS) in place, nor 2) that if there is such a mechanism, that is needs or will be applied to ALL blocks of flash memory.

Whether it is necessary or even helpful to employ an autoleveling feature that continually remaps memory when managing flash memory is really a function of the use model. What is REALLY nice to have is for remapping be available in the event of a bad chunk of flash so that it can be worked around. That's super helpful. Only requires that the system test flash for any goofs and have a bad block pool of spare flash available for the purpose.

100,000 erase/write cycles is a bunch. Truth to tell, it's a LOT more than flash used to have, and probably an understatement of the performance of most flash today. Some of them push 1,000,000 cycles.

But at the current state of the technology, assume 100,000 without leveling or any other mechanism in place to avoid killing it. Over the course of 10 years (a good bit longer than the expected product life of just about anything you can buy) that comes to 10,000 writes per year, or about 28 writes per day. Considering that there is no need for the TomTom to write to flash unless it is being truly powered down, changed data will be retained in DRAM until then. So you'd need to be doing a full power down and cold boot on a TomTom 28 times a day to hit 100,000 over the course of 10 years.

I wouldn't worry about it.

TomTom could have some flaky flash, they could have flaky processors that area having problems selecting or addressing or getting data from it -- but I doubt they're having any trouble killing it due to overuse, especially in the "boot" area. There are just far too many reasons why that shouldn't be the case.

 

[sageuvagony]

The boot loader is nothing more than a BIOS I would assume, like a PC. Just like you can format a hard drive on a PC. The PC won't boot, but at least the system will attempt to boot and can be recovered.

As far as the TomTom writing to the memory in a sequential pattern vs scattered, that would have to be a function of the TomTom itself. As we all know, the FAT system is a messy system and has the tendency to scatter files. Most SSDs (Solid State Disks) automatically evenly distribute information across all memory blocks. The same block won't be used again until all other blocks have been written to/used. At least that's what the manufacturers claim. We can only hope that's true for the TomTom itself, or else our bootloader memory allocation area will become worn out pretty quick, as well as the mapsettings.cfg file just like other members here have mentioned.

As I've mentioned here, I only run my 720 from an SD card. EVERYTHING resides on the SD card. The Internal memory is mainly used to store multimedia files on my device. If anything is going to die first, it will be the SD card. Those are inexpensive to replace.

I wouldn't be surprised if the boot loader resided on its own chip like a BIOS chip holds BIOS software.

From what I understand, you guys are saying that the internal memory might possibly be partitioned somehow to where the boot loader has its own dedicated chunk of memory blocks and cannot be accessed by user? Would this mean that the boot loader ONLY has access to JUST those memory blocks and can wear out faster?

 

[canderson]

From what I understand, you guys are saying that the internal memory might possibly be partitioned somehow to where the boot loader has its own dedicated chunk of memory blocks and cannot be accessed by user? Would this mean that the boot loader ONLY has access to JUST those memory blocks and can wear out faster?

To answer the last - no. That area would never need to be rewritten unless the bootloader section of the firmware is being changed. Reading it places no stress on it at all. The bootloader spends its time reading and unpacking stuff to DRAM, not NVRAM. No need for it to write anything to its own area at all.

I'm still not convinced that there isn't a separate BIOS and "bootloader" section, and I'm not convinced that there is, but I still can't IMAGINE that what we are calling the bootloader around here, something that actually gets reflashed now and again as part of a major firmware change, could be the same piece of code that is responsible for uploading new firmware. A company that allows customers to reflash firmware is asking for a lot of returned bricks if the code that does the uploading of the new code is allowing its own NVRAM image to be overwritten. Drop power and poof... dead device, and no means for recovery by the customer. The fact that we don't hear about that here is what leads me to suspect we're talking about two different animals.

ttsystem is the Linux kernel and the application all rolled up in one file. The "bootloader" unpacks the OS and application from that file in NVRAM to DRAM where it all runs and kicks things off. When a new application comes along, a new "bootloader" may come along with it on the front end of the package as received. Normally, the content of a file like ttsystem (received as part of the .cab of a new firmware release) is loaded into DRAM and checked for validity (CRC or whatever) before allowing it to be written to NVRAM. That's at least a safe approach. But a juice failure during the subsequent write to NVRAM, if the code that does all of this is included IN it, is a recipe for a train wreck.

I VERY strongly believe (and hope!) that there is lower level code yet that is capable of USB management and has the ability to pull in new firmware to the device and store it to NVRAM such that even a bogus bootloader load won't brick the unit. That could very well be a separate BIOS having nothing to do with the bootloader itself.

Anyone here familiar with something called UBOOT?

 

[gatorguy]

I have now. I can't resist an invitation to research