Ask HN: Can anyone recommend a Windows Systems programming book?
There's a hundred of these and I want something well-written. I'm primarily using Nim but a C/C++ book wouldn't hurt. Ideally it would have lots of exercises. The reason you're not getting a good answer is because the ultimate answer is to just use Microsoft's Documentation: https://learn.microsoft.com/en-us/windows/apps/ https://learn.microsoft.com/en-us/windows/apps/get-started/s... https://learn.microsoft.com/en-us/windows/win32/api/ It covers near everything, is extremely exhaustive, and constantly updated. That being said, if you're more interested in how the Windows API is organized/works internally (why you have to give it handles/resources and what those mean, for instance), then Charles Petzold's series is generally considered the definitive resource: I really miss the MSDN CD-ROMs. They had all of the information without network latency. Indexed and searchable. Categorized in a tree, complete with code examples you could immediately try. Press F1 on a function and there is the reference. Most of the old APIs still work exactly the same so I am frequently tempted to look stuff up in my VS6 install just because it's so fast and convenient. MSDN_Library_October_2001 is the latest one they mailed out for Visual Studio 6. you can download MS documentation via visual studio installer. Under "Individual Components", it's called "Help viewer" Thanks. I didn't know that. I've got Visual Studio.NET 2003 on CD, and it comes with a slightly more up-to-date MSDN library. I wonder what the most up-to-date version is. I remember reading a funny story somewhere recently that back in the day, ms developers intentionally left gaps in the docs or straight up didn't document certain apis etc so they could put them in their books and sell those so if you were working on certain things you'd be forced to buy said books. Back in the day, Microsoft shipped a figurative ton of documentation. I remember my dad coming home with this monstrosity: https://www.pcworld.com/article/917478/microsofts-biggest-so... I really wish we had these manuals nowadays. However we are in the mentality of fast iteration so it's probably a bad idea. A lot of this is there at learn.microsoft.com Yeah, but holding a book feels better. Maybe I'm just old style :/ I printed myself a few manuals. That’s nothing! When I was an IBM/Tivoli administrator, they’d ship about 12 linear feet of manuals every 9-12 months. Haha, that's totally believable and in tune with the MicrosoftMentality of those days. I'd love to see the corroborating evidence, though! If I recall correctly, the (also paid) MSDN developer reference documentation was actually pretty OK. The problems you needed Petzold and Richter for were the higher level "why is Windows this way?" topics and "what's the Windows-way to design XYZ?" questions. The actual APIs themselves had good documentation--the problem was knowing you had to use a particular API for a particular problem, which is what we needed the books for. There's the rather famous AARD Code (https://en.wikipedia.org/wiki/AARD_code), though that is from an era a little before Win32. And, well, it's not a documentation issue for app developers, as much a way to trick OS developers. LOL @ those memos. This was long before all our companies adopted the "Don't say anything in E-mail that could become evidence" training. Their ruthlessness and obsession with destroying everything not-Microsoft was truly unique and special. I can't think of any company here in modern times so focused on not only winning but making sure absolutely everyone else lost. Old-school Microsoft was wild. I know there are many legends like this, but I wouldn't be too sure about that. For example, most of the authors of the old-time classics were not employed by Microsoft when they wrote their books. I think Charles Petzold (Windows Programming) was a freelancer, Jeffrey Richter (MFC) was working a company called Wintellect, David Solomon (Inside Windows NT) had a seminar company (Solsem), I think initially primarily for VMS and Mark Russinovich (Inside Windows 2000) at Winternals Softwre. I've been digging for the article since morning, iirc it was something around directx, will update this comment when and if I can find it. It's not this link but it also touches on some of the stuff on this node https://retrocomputing.stackexchange.com/questions/25955/did... I'm convinced this is true for SharePoint development. There were things that I'd find in books that weren't even mentioned anywhere on any microsoft website, let alone documented properly. I'd argue that's how all modern "cloud" services are, some cool tech that Microsoft purchased and then integrated with duct tape and chewing gum. SharePoint, Exchange Online, Azure, it's all bound together by a string with APIs that can barely speak the same language. I remember a different reason - APIs were intentionally not documented so Microsoft Products were the only apps that would know how to use them (and gain an advantage over the competition) When I joined MS my boss told me that Microsoft apps weren't allowed to use internal APIs, I think as part of the anti-trust settlement. Windows components (e.g. the Settings app) can, but actual apps (e.g. Word) couldn't do anything third-party developers wouldn't be allowed to do. Some APIs are considered public but their documentation is less than useful, admittedly. But with those I don't have access to anything better either. :/ (Disclaimer: I speak for myself, based on my memory of public knowledge.) heck I would do it just for the perk of having to do less documentation at work Wow this is the first time I've seen someone recommend a microsoft.com domain name for 'updated' information. Usually its 1 year and 15 upgrades outdated. Its basically why I have completely given up on Microsoft. Every update is a breaking change that isnt updated in their docs. And chances are, it wasnt going to work anyway. Any time you have to use a microsoft service/product, know that documentation is just going to be worse than any FOSS made by teens. I have no idea how you got that idea about Microsoft's developer documentation. It's always been one of the more robust options out there. You can find full tutorials and documentation on the entirety of WinUI3, which is the literal bleeding edge; for instance. You will also often see big chinking warnings that the site is no longer updated pointing you to homepage of current docs site, despite the APIs you're looking for being still current. Is the new documentation lacking something you need? Is there some reason you don't use the outdated documentation, if it's working for you? Are you mad because there are redundant resources available to you? I'm literally writing this to you on a Laptop exclusively running Fedora Linux. I do most of my development in OSS tools. I'm not some fanboy, yet despite that (or maybe due to that) I'm still able to recognize the very obvious fact that Microsoft has very robust documentation. It's a problem because it makes one doubt whether they are in right place. What's more, it's a step back from how well Microsoft did it in the past. It's essentially like stepping into half-done move between two different places, and never being able to tell where did you put what. They can, and in the past did, do better. That doesn't answer any of the questions. That just introduces another vague problem of feelings. Perception is important in discoverability of documentation, and yes, vibes are also part of it. The complaint is that now it often ends up disjointed, especially when you're starting to search for a topic, and the links pointing you to "new site because this one no longer updated" don't point to the same topic (or area) on the new site. Also, I have absolutely no idea where you come from with fanboyism or Microsoft hate as context in previous comment. The Windows Internals series is typically the go-to for what you're looking for. Combine with MSDN browsing to determine what you want to do and how you want to do it. Agreed, also anything else from Russinovich. https://learn.microsoft.com/en-us/sysinternals/resources/win... But you have to be aware the part 1 is now a bit dated, the 7th edition is from 2017. The part 2 is from 2021. It’s not a big deal but some references are a bit outdated and some modern stuff is missing, but nothing that will be a blocker. The books aren’t focused on programming though, but they explains the security, threading model, etc. Really interesting reads. I comment it often but something great with windows API is that you can still use all the Win32 APIs, almost all old examples you can find on MSDN are still relevant to this day. The only thing you need to get started is a way to interact with C, and learn windows conventions (parameter names are weird at first but make sense, the error handling can also be confusing at first, the process model is a bit strange at first if you come from Unix, etc) MSDN and that book are great https://learn.microsoft.com/en-us/sysinternals/resources/win... A lot of useful links on that page for additional reading. Repo for the tools used in the book https://github.com/zodiacon/WindowsInternals which are invaluable That's the answer. Windows Internals is a really good source for deep understanding of the OS. The old new thing[1]. It's a blog, but he did also publish a book[2] compilation of some choice blog posts. The book is from a few years ago, so some specifics might be outdated, but I think a real strength of the blog is how, despite being very targeted, focusing on specifics of one OS, it's also very general. I also just really like Raymond Chen's writing style. I agree that this is a fun book/blog, but I would recommend anybody trying to learn Windows system programming to look elsewhere. This is more like a collection of fun bugs and weird design decisions. True, it's not something you would sit down and read through. Subscribing and passively consuming will filter knowledge in over time though. I keep my copy of the physical book in my bathroom :p Programming Windows - Petzold Just a warning, that the very latest edition of this book is a complete rewrite for WinRT and Windows 8. It’s still a great book, but you’ll probably want the previous edition to learn how to code traditional Win32 programs. With a caveat, even though most of us have stop caring about WinRT stuff, that is still how some APIs in Windows 11 are now being made available, with the difference of still being WinRT, without the UWP infrastructure around it. Example, the new MIDI subsystem, https://github.com/microsoft/midi This is the best answer. Windows Internals is a great book/reference for understanding how Windows work, but it won't teach you how to actually program Windows. The Fifth Edition of Petzold's book will actually take you through the Win32 API, which is still usable even on modern Windows platforms. Programming Windows and Inside Windows are the books you want. Programming Windows covers systems programming in C and in later editions, in C#. Inside Windows goes over the Window OS architecture in-depth. There’s not much programming in the book but you will come away with a deep understanding of the OS. Both books are published by Microsoft Press. You’ll of course want the latest editions but if you come across earlier editions, those are fine too. I’m partial to the earlier Petzold editions of Programming Windows. The technical docs at microsoft.com are also super helpful and up-to-date: https://learn.microsoft.com/en-us/docs/ Off topic maybe, but would like recommendations for macOS too, especially C/C++ given that everything is Swift now while I suspect that even so major apps are not using Swift. Windows has much more available.[1], [2] [1]:https://learn.microsoft.com/en-us/sysinternals/resources/win...
[2]: https://www.amazon.com/Windows-Kernel-Programming-Pavel-Yosi... Interesting question! You might want to consider starting a new thread for that. I would recommend "Programming Server-Side Applications for Windows 2000" by Jeffrey Richter and Jason D. Clark. Richter also wrote "Windows via C/C++", which was previously titled "Programming Applications for Windows 2000" (4th edition) or "Advanced Windows" (3rd and earlier editions). I realise these books are now very old (17 years old for "Windows via C/C++" and 25 years for "Programming Server-Side Applications") but really, the Windows API hasn't changed all that much in that time. Besides the great learning resources at MSDN, the Windows Internals series is a requirement. Then depending on which area of Windows you want to focus on, there are plenty of Microsoft Press books. Hello, Windows via C/C++ by Jeffrey Richter is what you are looking for. Yes, it’s a little dated but it’s a a great book to start with. What kind of systems programming are you interested in? A sibling comment already mentioned Windows Internals, but that's really about the architecture of the Windows kernel. If you want to write applications, you probably want to read something about the Windows API? What exactly do you consider systems programming, i.e. what would you like to achieve? Windows is not open source, so unless you work for Microsoft, you won’t really be doing much “real” systems programming probably. Do you want to program applications? Windows services? Device drivers? Considering the absurdly huge number of hardware devices that have windows support, SCADA and automation system that run on windows, lab instrumentation, security and compliance software, I wouldn't be surprised to find that there are more system programmers making a living out of windows in a professional setting than linux system developers. Well that’s a fair point… I guess you could look at things like antivirus software as systems programming probably (or both systems and application programming) Do you think that "real" systems programming only occurs at the kernel level? No, do you think operating systems only consist of a kernel? What would you consider systems programming on Windows for example? A task manager application which tells you how much memory your applications are using Ok I think I would consider this one application programming. But I can see how one could have a different view. For the people here who know windows and Linux, what are some of the major differences between the two that you like/dislike? I know nothing about windows internals, but I’m curious In Linux, everything is a file / a file descriptor. In Windows, everything is a handle (except sockets, which are copied from BSD). In Linux, configuration is done via plain text files. In Windows, there is the registry. In Linux, there is the root filesystem, where everything else is mounted beneath. In Windows, there is C: and more. In Linux, the native string encoding is UTF-8 code units stored in char of one byte size. In Windows, the native string encoding is UTF-16, stored in wchar_t of two bytes size. In Linux, there is fork. In Windows, there is no fork. In Linux, there is no unified graphics API. There is GTK and Qt for UI, and OpenGL and Vulkan for games (roughly speaking). On Windows, there is Win32, WinForms, WPF, WinUI, DirectX and also OpenGL and Vulkan. > In Windows, everything is a handle (except sockets, which are copied from BSD). A handle to an object. I think that is the main difference that is also visible in tools like PowerShell (as compared to the UNIX shell). Linux is very free form, Windows is much more structured and strongly typed. I'm not sure I understand, the APIs are opaque HANDLEs, nothing prevents me from passing a file handle to ShowWindow(), just as nothing prevents me from passing a timerfd to write(), of which neither makes any sense. PowerShell is .NET and operates a few more abstraction layers above the Win32 API. HWNDs are actually a bit special. HANDLEs you get from CreateFile and so on are implemented by the NT kernel and are real capabilities (like Unix fds), so the actual bits of a HANDLE from one process are useless in any other one. On the other hand, HWNDs are global (at least within a window station, I don’t know how that works). If you display the numeric value of a HWND in a message box in program A and type it by hand into program B, program B can (usually) pass it to PostMessage and successfully reach program A. I don’t know if other USER and GDI “handles” (e.g. HMENU, HRC) are like that, but I wouldn’t be surprised if they are. I also think defining STRICT before windows.h will still prevent you from passing a HANDLE where an HWND is needed, even though the last (also the first and the only) time Microsoft promoted this feature was in the Win16-to-Win32 migration docs in the early 90s. A HANDLE to a file vs a HANDLE to a named pipe or even a mutex is another matter—those are all genuine NT objects, and handles to them are to an extent all the same kind of thing. The Windows APIs like types more than POSIX, which likes to use integers for everything (though windows uses handles a lot) Also The Windows API pattern is typically: 1. Initialize a "descriptor" struct 2. Pass that descriptor struct by reference to a "create" function 3. Get ComPtr<> for resource Where a ComPtr<> is a smart pointer that manages the lifecycle of an object across different processes Also Windows built UTF-16 support into their APIs 30 years ago, so a lot of API calls involve constructing UTF-16 literals Anything by Matt Pietrek and Mark Russinovich. Windows Internals, 7th Edition. K&R .NET has been the standard interface at Windows system level for many years. The ".NET languages," particularly C# will provide much less friction than .cpp (or Nim). For no other reason than the quantity and quality of tutorials and documentation written by technical experts and edited by technical editors who are getting paid to write and edit from line items in Microsoft's regular operational budgets. Or to put it another way, picking languages takes exploration of Windows system programing from an X problem to an XY problem. Good luck. Not sure if the systems programming mentioned by the OP includes drivers, but AFAIK the standard language for drivers is C/C++. Not that it's impossible to use C# (see https://github.com/ZeroLP/WDK.NET), but it's not necessarily the most widely used approach. You're recommending that OP uses an abstraction over the system. It takes their X problem and makes it a Y problem instead (because they won't learn systems that way)! Depends on the objective of OP. Abstractions always leak some of the underlying capabilities so you could use them as a more beginner friendly approach to get acquainted with the system while doing something useful or fun. For related examples, you can learn Ruby just by learning Rails. Or in more general aspect, the web technology stack through a modern web framework. Not entirely true, Native Interop is pretty good with DotNet https://learn.microsoft.com/en-us/dotnet/standard/native-int... Though I believe it has regressed at least a bit with the introduction of DotNet Core. Yes, but he or she is giving convincing sounding reasons that might be helpful for the op. Maybe it is actually better for OP to learn .net, maybe not. But the answer had quality, so I do not think the downvotes are appropriate. While I think the intentions of the voting system are for marking low quality, as you've said, I think in practice most downvoters are using it to mark their disagreement. There's something interesting about that; it implies that grey text signifies a non-conformist view and approach to a technical issue. As is the case here. I like to skip to the grey text. > Please don't comment about the voting on comments. It never does any good, and it makes boring reading. From a technical perspective I think my comment is interesting for anyone running a platform. I don't think this guideline applies. I wrote the downvoted comment and am a proponent of downvoting for disagreement. Downvoting is an alternative to arguing and often makes for better comment pages. It's a great perspective. In fact I upvoted your comment just now. By contrast, it also prevents users from commenting. See dang's posts on this. Regardless, it's interesting to note that it also, indirectly, highlights interesting comments. Do you also find that interesting? Typically, if a comment has been downvoted, I use it as a reason to filter it out. There’s more comments than I am going to read so reducing the pool is useful to me…and I don’t identify as a contrarian. Thinking about it, downvoting for disagreement is the analog of upvoting for agreement. I don't think being contrarian is a requirement for seeing value in comments that others disagree with. Just on this thread, your comment was downvoted but represents a particular way of solving a problem that differs from the downvoter's. Even if that way is not better, for them, I think acknowledging that it might be better for someone else is objectively open minded, and not in the least contrarian. But my point isn't to criticize anyone for downvoting. Just the opposite. It's to illustrate that thinking outside the box is in itself disagreeable, which, as is the case here, can be a searchable metric. I can see why your approach is useful for you and probably for other people. I just don’t seek out gray comments (though sometimes I do read some) because I am ok with missing interesting viewpoints. HN does fine despite my behaviors. I am not contrarian in the sense that I think good comments are downvoted to a meaningful extent. YMMV. Depends what you mean by "system", but there's plenty of the Win32 API that's not exposed by first-party packages in dotnet (e.g. audio). It's certainly the most convenient way to write an application (once you get over the UI toolkit disaster area), but "systems" programming might also include drivers. If someone is hiring Windows-systems-programmer-1 it is more likely to be about performance of .Net applications than writing drivers. And sure things like managing thread pools and tweaking garbage collectors and allocating resources in general are less sexy. So reading about writing drivers and talking about writing drivers and watching YouTube rants about writing drivers is more entertaining. The OP is coming all the way from Nim into two decades of .Net as Windows “easy mode.” There are important reasons to use cpp, but most of the reasons in the world of Windows are .Net.