Logging data before NTP time sync established
- with discharged RTC battery, or without RTC
Concept applicable: '''Raspberry PI, Orange PI (see BuildrootOrangePiPc
), or other embedded systems'''
- even those that can only run C or Lua (and not busybox /bin/sh and awk as in the examples)
- more and more systems are mobile, and network may or may not be immediately available when booting in a tunnel, or elsewhere
Waiting indefinitely until NTP sync is established is brain-dead !
This is a real existing problem (that is often not solved or solved terribly, not in a robust way) effecting the usability and maintenance cost of millions of "IOT" systems (soon billions). Several nice ntp hints above, NTP is useful, but sometimes comes late to the party:\nÿ1ÿ
Marcell's method: subtracting uptime (first field of /proc/uptime) from unix date ($DX) = is the (DB) boot time (might theoretically change by 1 because of rounding, never seen that though). Some recently recorded example filename=date...$DB.$$.$c.myapp.gpg:\nÿ2ÿ
Even if NTP manages to sync after days of datalogging (network problem), the logged data is safe, and easy to recover timestamp automatically during data presentation/analyzis, by detecting $DB jump because (at the time) of NTP sync. Even if NTP never syncs for some reason, data will be logged, and consistent (though exact time offset cannot always be recovered in that case; time-sync can often be established from the logged data - eg. light sensors, or events/sounds also recorded by other systems).
Note: the $$ will likely be different after boot, and $DB will be different as well, $c counter confirms continuity.
It is our task to implement robust systems that work reasonably (baby baby it's a wild world) in as many
situations as possible (consider that more and more systems are mobile, and network may or may not be immediately available when booting in a tunnel, or elsewhere).
It's unprofessional to implement fragile systems, or let issues escalate instead of containing them (1 subsystem does not work well, eg. network or NTP => other subsystems also freeze => such practices applied in large systems make the right recipe for circular dependencies, unreliable systems, or total system collapse at the worst time we can imagine). It's unprofessional to add unnecessary dependencies (=let someone else, eg. administrators solve it, we are programmers and just don't care). Eg. requiring that network and gateway and time servers are always up and reachable right after boot, and RTC installed and maintained (replacing batteries possibly in dozens or hundreds of devices). Yes, of course reliable network and NTP systems are welcome. Battery RTC in every gadget is usually more pain than gain.
In general, data should be logged (instead of freezing the system unnecessarily) in some usable way if at all possible.
Further improvements are possible (also, porting /bin/sh to .py or .c or ... ), of course.
- Usually logging in UTC, display in TZ localized form is a good idea (local-time can jump back, which can sometimes cause problems).
- Bash is awesome, but it is kind to avoid bash syntax, if it's easy to make it work well with /bin/sh dash or busybox shell...
- avoid the bash "function" noiseword.