Ed Yourdan, eh? That's a name I haven't heard in a discussion since the mid '80s.
I haven't been a professional programmer, in the last couple of decades of my career, but I was one for about 25 years. Interesting "good documentation" is still a question, but yet, in a way, not surprising.
In my professional programming experience, I've see the two extremes, i.e. documentation that's worthless (and which can be a time killer trying to sort out the chaff from the wheat) and no additional documentation whatsoever (also a time killer, but at least you don't have to figure out what's good vs. bad).
Laugh, for the latter, my "ghoulish" camp fire story, was when while working at a major regional bank, they said, we understand you can program in 360/370 assembler, understand different file accessing systems, and on-line too, correct? I said yes. Well, we have this, in house written, commercial loan system, we would like a few changes made to. Okay, where's the code and documentation and what do you want changed?
Well, there was no external documentation, at all! Since they asked about 360/370 assembler, it was, unsurprisingly, written in 360 assembler, 10,000 lines in one program. But, it was 10,000 lines of assembler code without any embedded comments or even symbolics, at all! (Basically, if you've ever seen a complier generate "machine code", it looked much like that.)
e.g.: MVC 123(45,6),78(9)
i.e. MVC D1(L,B1),D2(B2)
move characters at address base register B2 + displacement D2 to
address base register B1 + displacement D1 for the length of L
(Oh, if you're wondering how you lose source code, these programs dated from source code likely being on punched cards. Not all that hard to lose the one and only copy of a card deck you're not actively maintaining. Just some minor differences in computer development environments in the early 60s.)
But wait, there's more. It was an on-line program, that generated the terminal control codes for terminals that no longer existed (for years), and which there was no documentation for, and it used a home-grown file management system, that also no longer existed (for years). If you were wondering how it could still be used, years prior, some system programmer wrote a translator to take the out-of-date terminal codes and map them into the current on-line monitors, and another translator to take the non-existent file management calls and map those into VSAM calls. (Of course, any documentation for either translator didn't exist, either. Don't recall if they even still had the source codes for those. [For another old program, it used a subroutine they definitely no longer had the source code for. To use it when you updated the main program, you had to use the linkage editor, to link/edit in the object model from a prior load image of the combined programs!]) Yea, it was "fun" making changes, which I did.
Conversely, I was working as a programmer, and while in a "bull-pen", one of the other programmers exclaimed "who the duck wrote this"! (NB: "duck" wasn't the actual word, but let's keep our PG rating - laugh.) Next he said, "Oh, Joe, you wrote this."
I said, "What's the problem?". He responded, "It's the first time I've seen code, written by someone else, I can easily understand!"
Once upon a time, someone floated/suggested the idea that code should be "self documenting". I agree, although, often you do need some external to code documentation too. (Well that, and the fact, very, very early in my professional programmer career, I wrote a small [100 to 200 lines] program - worked exactly as needed. Two weeks after it went into production, I was asked to make some minor revision. I couldn't make any sense out of what I wrote/coded, myself, just two weeks earlier! This made quite an impression, how to write functional, but unmaintainable, code! Had to completely rewrite the program. I vowed, never again.)
Basically, when writing code, my style and usage of variable and/or function/procedure names tries to provide as much information as possible. For example, might I write function abc(parm), literally as function "abc"? Anyone consider that good practice? If you did that, might you want to comment on what this function does?
But, what if the function was "sr", that's better, right?
No? What if the function was "SquareRoot"?
Now the forgoing is a trivial example, but of course, "SquareRoot" is too verbose, right? So is "SqRt" okay? Well, personally, I would likely use "SquareRoot", but then I know how to touch type and/or I later do a global edit replace, maybe "sr" or "sqrt" with "SquareRoot". ; )
For explicit embedded comments, I rarely comment on the "how", as the code shows that, but on the higher level of "why", especially in the goals of the system and/or program.
Generally, the goals of the system (like design requirement), would be the most likely thing that would/should reside in external documentation. If placed in programs, often much replications, if all the system tries to be documented there, however, goals specific to that program, might be well documented as part of the program, and those external to the program, might be referenced by naming other programs that this program interacts with.
In my experience, program code, over years, doesn't seem to get lost as easily as external documentation. Further, you might find it a little easier to get programmers to update embedded documentation, within the code, if code changes impact it.
So, to recap, what I found works well, i.e. work toward "self documenting" code, and try to also embed the "why" in the code.
Is the forgoing perfect? It's not! Just trying to find some workable middle ground from nothing beyond, effectively, "machine code" to code documentation like:
a=b ; load variable a with the value of variable b
or
if a=b ; if variable a equals variable b
That aside, I did a quick look at your references, lots of good suggestions there too.
Basically, though, one-size doesn't fit all. Documentation can be tailored, assuming certain expectations. For example, within the function "SquareRoot", I likely don't need to document what a square root is, although, perhaps you might mention a particular published algorithm you're using, or something like, you may get slightly different results if using words sizes of 32 bits vs. 64 bits.
Conversely, though, for "JoesSnakeOil" function, you might need some additional embedded documentation about what it provides.
BTW, some of your Yourdan's quotes remind me of:
One project, where I wrote a 10,000 line Cobol program; took me two years.
Two interesting "things". First, the day, I found the code wasn't working correctly. It took me half a week to find, in the 100,000 lines of compiler generated assembler code, the 3 specific instructions that were NOT there. (Laugh - on the phone with the IBM support engineer, it took me about 20 minutes, to get them to understand what was NOT there. When they finally understood, they asked, "how did you find this?" Well, it did take some work. Interestingly, 2 days later they came back with the "fix". I asked, "you have a fix already?" They responded, "no, it's a known bug that was patched 5 years ago." [BTW, I hate bumping into known bugs that have been fixed!])
Second, several years later, I was talking to a programmer still working at the site, that mentioned they had one issue with my program. I asked, which is? They responded, it's the two page assembler subroutine you wrote and call from your main program, we wanted to move your program to a UNIX system too, but the 370 assembler subroutine is a "problem". I suggested, I'm sure a similar assembler routine might be written on whatever other platform you wanted to use, but if not, you can use the Cobol function my subroutine replaces. Remember the only reason I wrote the replacement was to decrease the memory footprint by about 75%, although, I didn't expect my subroutine to be about 3x faster too. (This subroutine was, hmm, perhaps unusual [the one "key" {to what I was doing} instruction I was using, probably most would say, why would you ever have a need for that?], so, I very much tried to document, within the two pages of codes, what the heck I was doing.)
