It's that simple in that, for any business network, you need to do just that: keep a record. It's not that simple in that, for most business networks, it's not mandatory to keep a record. Personally, I don't recommend using a log book as it doesn't allow for the inclusion of external documents.
If your company lives by paper record, you should be keeping a set of folders, one for each system. Entries should be made via a set of forms (incident, maintenance, configuration change, etc.) that can be dated and signed by personnel concerned with the specific evolution. For some of the entries, management should sign.
If you take the electronic path, I recommend a Wiki or even just a set of folders in a directory on a stand-alone system (not networked!). The same idea for blank form follows: keep a set of templates handy that you can cut-and-paste from.
In either case, you want to limit the access to the logs. If they're paper-based, keep them in under lock and key. If they're electronic, restrict access and don't network the system. File or file system encryption might be useful (if not time consuming). Side note: backups are your friend.
The entire point of the exercise is to produce a legally useable record. It's a benefit for the company in that it can be used to display due care (compliance). It's a benefit for you in that it becomes a reference for keeping track of who did what to when and when. It is valuable to anyone that follows you after you've moved on, so that they don't have to repeat your mistakes (yes, you should include them too) and it'll minimize having to figure out if you did or didn't perform a specific action on a machine.
I used the phrases "mandatory" and "due care" above to denote that there are now a number of laws (GLB, SarBox, FISMA, HIPAA, etc.) in existance that require due diligence (having policy/practices/protections in place) and due care (recording the exercise of due diligence). Most of those laws (if not all) don't care how you perform these functions, just that you have them. If you (as an organization) use a well-recognized set of practices (e.g., ISO 17799), so much the better. You'll use less time having to defend them, should you end up in court.