Robotics 101 at UMich: Applied numerical linear algebra as intro linear algebra
robotics.umich.eduYoutube playlist for the course:
https://www.youtube.com/playlist?list=PLdPQZLMHRjDK8ZbLIcq1Q...
Materials on Github:
I took this course 3 years ago. I found it fast-moving, and it focused a lot more on applications than fundamentals, which meant it was more wide than it was deep. This didn't turn out so well when I decided to study ML later and needed stronger linear algebra fundamentals, but it was a fun course. There were a couple interesting course projects, one of which was using linear algebra to balance a (simulated) 2D robot.
No one, and let me repeat that, no one "gets" linear algebra, differential equations, or frequency domain on the first pass. It takes years to absorb and multiple passes.
See:
Bruner / Spiral Curriculum.
Ebbinghaus / Spacing effect
Hattie / Deep-surface-transfer learning
Chunking ("How People Learn" has a good copy on this)
Etc.
The way you do this is you take a course, and then you take more courses. After a few years, it all connects and makes sense. The first course, I find, is often best short, simplified, and applied. Once you get through that, you can go deeper.
Different angles are nice too. For linear algebra:
- Quantum computing
- Statistics and probability
- Machine learning
- Control theory
- Image processing
- Abstract algebra / groups / etc.
- Computer graphics
All come to mind.
On a mile-high level, this course seems ideal for a first pass. On a detailed level, I'm confused by some licensing issues.
Not with the way it is taught. But if the course structure is changed slightly to have reinforcement of early concepts woven through the course, people learn much better.
At least that was my experience when I taught it. See https://bentilly.blogspot.com/2009/09/teaching-linear-algebr... for more detail on my experience.
> No one, and let me repeat that, no one "gets" linear algebra, differential equations, or frequency domain on the first pass. It takes years to absorb and multiple passes...
I don't understand the point of this comment. On the one hand you're trying to encourage people by saying "don't feel bad you didn't get it the first time" but then you throw a mountain more work/terms/books at them? You think it's encouraging to a student to hear that if they didn't succeed in this robotics class because the LA coverage wasn't great ...... they should go take quantum computing, control theory, abstract algebra classes?
Really for my linear algebra courses in pure math i was comfortable--but some applications courses would help me understand the usefulness.
Tangent, but how does that course make anything "more equitable" as per the video?
One of the umich grad school prereqs for economics was linear algebra, and it was literally just that - pure math.
Where do you feel the gaps were for what you needed for ML? Downthread, Jesse Grizzle notes they've added some stuff in 2023 (it's on Github I think?) to support an ML class.
What would you recommend for building a strong linear algebra foundation?
Also a big fan of Strang. "Linear algebra and its applications" has problem sets with solutions for odd number questions.
Would highly recommend https://mathacademy.com/courses/linear-algebra or https://mathacademy.com/courses/mathematics-for-machine-lear...
I originally spent time working through practice problems from one of Strang's books, now really appreciate how systematic math academy is in assessing, building a custom curriculum, then doing spaced repetition.
i don't really care how many people i respect liked it, i have to be honest, i hated strang's "linear algebra and its applications."
there's a strang text on computational science that was much more my speed (less of the baby talk and repetitive manual arithmetic exercises) and i think that some of the revisions that came later (+ "learning with data") were better.
i did not find doing endless exercises of gaussian elimination or qr factorization by hand on small matrices to be all that enlightening.
this michigan course looks awesome!
> less of the ... repetitive manual arithmetic exercises
I think this post (from a math academy employee) has a good argument for why these sorts of exercises are important. It's about basic arithmetic, but I think it applies to tedious things like performing gaussian elimination on small matrices as well.
https://www.justinmath.com/if-you-want-to-learn-algebra-you-...
I like to come at it from both angles - higher level with useful applications, and then lower level "I could maybe implement this if I had to" exercises. The latter are tedious, and hard to motivate effort for without the former. Ultimately, as the post argues, I agree that if you don't understand the lower level (tedious) operations, you will only get so far in your ability to apply LA.
I took 18.085 (applied linear algebra) as a grad student at MIT. The best taught math course I've ever taken. Strang is a fantastic teacher.
After working with math academy, any form of video learning seems so inefficient. I think people lose a lot of time watching these videos thinking that they are learning without applying anything by themselves.
It’s $50/month online course. As effective as it can be, I can’t justify this expense for myself, as much as I’m fascinated by math.
UMich has a couple other linear algebra courses that might be better for that: MATH 214, MATH 217 are the numbers if I remember correctly. 217 is known for having a high workload and greater rigor, but some say it's worth it even for non-Math majors.
In terms of books, I would say Linear Algebra Done Right. The book requires some background to understand efficient. But, once you have some background, it is very good for having a systematic and rigorous understanding of Linear Algebra theory
LADR is the SICP of linear algebra.
If you can handle it, fabulous. If not, you're really in deep doo-doo. There did not seem to be a half-way to me. Astounding exercises, and also some are astoundingly hard.
Anything by Gilbert Strang
Man this would have been nice when I was in school.
For some reason linear algebra still isn't part of standard Mechanical Engineering course load (Calc 1, 2, 3, DiffEq) which made life extremely difficult in some of the later classes. I remember spending weeks brute forcing a lot of things that would have been trivial with a little bit of matrix math.
I took a superficially similar class as a 400 level elective but it assumed everyone already knew linear algebra going in, and it was a disaster.
> For some reason linear algebra still isn't part of standard Mechanical Engineering course load (Calc 1, 2, 3, DiffEq)
Wow. In my undergrad all engineering majors had to take linear algebra (calc 3 was optional for computer engineering).
This would have helped me get an actual CS degree 25 years ago instead of CS-lite (networking & server admin).
It's not that I can't do calculus, I took it in high school, and then again in my first go-round in CS. It's that I hate calculus. Not the subject itself, just the grinding away at problem sets.
I did a refresher in pre-calc, calc I, calc II & discrete mathematics during COVID at the local community college (was planning to finish the few credits I need for an actual CS BS) & I started calc III twice (but dropped both times). I even got a 4.0 on my first calc III exam (and this was an in-person class, so no online shenanigans).
I just have some kind of weird aversion to 3 dimensional calculus. I have convinced myself that I'm simply not smart enough to actually do the work. I understand it, I just get clammy with it.
Truth be told, maths are my kryptonite. Despite working with numbers all day every day for 30+ years, and writing a lot of software over the years (and not just CRUD, but games of all things), I am absolutely ashamed that I just can't seem to grok math with any rigor.
I have all the Stewart textbooks on my shelf, many textbooks from libgen (ones I've seen recommended on HN from people who went to much better universities than I attended), and I even work through problems a few hours per week. I just can't seem to make that leap from a guy who's "good with numbers" (from a layperson's perspective) to a guy who's good at math.
Maybe I need to break open one of my physics textbooks and actually use the calculus in an applied context and that will break whatever mental barrier I have (I've even watched all of the 3 blue 1 brown videos, countless youtube lectures, etc).
A few hours per week simply isn't enough, the best success I've had studying was 6 hours a day, resting for 3 (leisure activity), "working" for 3 (class, commute, chores, related reading) and sleeping 12 hours.
In books like Stewart, staring at a theorem until you can write it's proof should trivialize most problems in the book.
If a method for solving a particular problem is too difficult for you maybe consider researching and/or inventing some new method to solve these problem. People created these methods in the first place because earlier methods were too tricky
Or just focus on work that doesn't require hundreds of hours to gain proficiency. As long as you have time every day to stop, think, and come up with an idea that solves a problem you won't become intellectually unfit.
Maybe your problem is Stewart? I used that textbook and was successful, but it's not for everyone. For example, beginning calculus with limits is another bit of misguided conventional wisdom. I still don't get limits, really. Serge Lang's calculus book takes the approach to just roll with an intuitive notion of limits, saving rigor for analysis. Which seems better.
Gilbert Strang has a textbook, also more intuitive and applied. Free PDF provided by MIT. Sylvanus Thompson's book is recommended here, again, intuitive, applied.
Other comments here, 3 hours isn't enough, use Math Academy, nobody gets it on the first approach, all seem relevant. One of the textbooks recommended here says in the preface that it's for a second course in Linear Algebra. Analysis is just calculus the second (or third) go round, and it's said to be the hardest class in a math major.
I am in your boat, but about linear algebra instead of calculus. This is what I try to get myself over the hump.
Thank you. I appreciate the feedback and the pointers.
I honestly don't think the problem is the textbook.
I have Stewart (both the standard version and Early Transcendentals), and I also have a book from 1967 by Tom Apostol (the 2 volume set that covers single & multivariable calculus, linear algebra, a subtle introduction to differential equations and some probability as well).
My gut feeling is that I just don't know the correct way to study math in general. I have no problem doing the work. But it feels more like mechanical or algorithmic solving than it does like true understanding. There is a difference. I can't deconstruct a problem and think in the abstract to come up with a different method to solve it.
And there always seem to be some fundamental truth that I'm always missing. A part of a proof here or an axiom there that seems obvious to other people who study these subjects that I just don't "see".
It's incredibly frustrating, because deep down I know I have the aptitude for this stuff. I guess that most subjects have always been easy for me. I could ace exams without cracking the book (or just skimming).
Math is not like that. You need to read. And then re-read. And then do. And then do some more. And then go back and re-read again to see what you missed. And there's a lot of things that are between the lines, and if you're not following it, those things fall by the wayside.
I just need to learn how to learn math. I need to learn how to deconstruct notation and proofs to truly understand them. And there's no shortcut. It's grind and grind until it all becomes clear. That sort of thing is just difficult for me.
I feel what you're describing very viscerally. I have tried so many times as an adult to finally get linear algebra. Worked my way through Strang to eigenvalues and eigenvectors repeatedly. Still feel like I am failing to see something.
---
I hear Apostol is hard. Like Spivak-level hard.
> It's that I hate calculus. Not the subject itself, just the grinding away at problem sets.
Most math majors I knew hated the standard calculus courses, for precisely this reason. It's taught this way because they're targeting engineers and some hard sciences (physics).
The reason is that for many of those majors (EE, physics), you will take courses where doing calculus is your daily bread and butter. You need to be as adept at it as algebra. Over 50% of HW problems in those courses will involve calculus. They really don't want students who understand circuits but can't do anything useful because they stumble on calculus.
They are the largest "customers" of the math department, so the department caters to them.
I swapped out from being an Engineering major (consistently in the top 5 scoring Engineering students out of a first year intake of 300) to being a math major for similar reasons - Engineering has a lot of rote learning grind exams, not so much exploration of deep fundementals.
The Math dept has numerours courses loosely covering similar material, Math 100 - first year math for math nerds, Math 110 - first math for engineering students, Math 120 - math for business majors, etc.
Math for math majors ( the 100 stream ) had 20 students in all (IIRC) most of whom now hold academic positions, Math 110 had the 300 engineering students, other streams had cross over students from business, medicine, law, et al.
Future mathematicians (and theoritical physicists, etc) are indeed the smallest group the Math Dept. catered for.
There are deep concepts behind multi variable cal. But if you just want to pass the course, memorizing problem shapes through practice will get you through calc 3.
Do all the homework problems check the answer in the back of the book. You’ll make it.
One day, God willing, lol.
Maybe stop torturing yourself for a bit. Sounds miserable.
Such is life. We all have regrets, and being sloppy and indifferent with math in my youth is a big one for me. If I had done it then, I wouldn't have to try so hard now.
You should check out Math Academy.
It schedules everything for you including the review so you just have to keep showing up to do the work.
Even as little as 30 minutes per day done consistently for months will have you make tremendous progress.
And once you master multivariable calculus, fields like probability and machine learning will be unlocked for you.
If by 3d-calc you mean vector calculus then yeah I never actually understood any of it, I just moved on to differential forms and the tensor calculus and Riemannian geometry and then wondered why anyone bothered with 3d-calculus in the first place.
Most of the time I've found that the deeper I plunge into abstraction in math I get rewarded with an extremely elegant formalism. Its like upgrading your weapon in dark souls, the early game enemies get one-shotted when you go back.
Maybe some schools do but it's baffling to me its not a universal requirement. It'd be dramatically more useful than Calc3 for most engineers.
Michigan doesn't seem to require it as the College of Engineering core classes or as part of the BSME (checked because they're who this course is through):
https://me.engin.umich.edu/academics/undergrad/handbook/bach...
And my alma mater has a very similar progression.
First year linear algebra is a requirement in Canada for national accreditation of any engineering program.
Linear Algebra was a first year subject for me as well. I studied Engineering in Australia.
The Robotics concentration requires Linear (Matrix) Algebra.
And some upper level courses have a prerequisite.
But indeed it does seem a avoidable for many MechE majors.
Funny thread about UM engineering students avoiding taking UM math classes. https://www.reddit.com/r/uofm/comments/15w18gv/reminder_you_...
I did an EE/CS dual degree and there was a really interesting difference between the linear algebra courses offered in both departments.
For the EEs we were given a crash course in GE120, which all engineering students had to take. It covered how to use determinants, Gaussian elimination and matrix inversion, and those kinds of “basic” LA tools, plus some simple numerical methods stuff like Newton’s Method. In second year we had a short lab course that focused on how to use Matlab, and a circuits analysis course that pretty much forced us to learn how to represent large sets of equations in matrix form and invert them to solve all of the variables at once. Very very practical.
And then in third year I had to take a 200-level linear algebra course from the Math department to satisfy the requirements for the CS degree. I chose the honours version of it and… holy moly. I thought it was going to be a gimme class but it turned out to be very theory-heavy, of which I had learned almost none in engineering. The first month kicked my ass pretty hard. Once we got out of the low-level theory (which was truly amazing to take in) and into the more advanced things that I’d been using for 2 years but didn’t know “why”, everything changed. Many of my peers were struggling to understand why you’d want to do some of this stuff and I was just super excited to finally understand why the “just turn the crank” math I’d been doing actually worked.
We had the basic Linear Algebra in high school
Yes back in 2005 when I first went to undergrad as a mech engineering major, linear algebra was not a requirement. Our mechanics professors were highly irritated by this.
I don't think this has changed much (but absolutely should). I've watched in real time as Micron representatives reject mechanical engineers and prefer résumés from industrial engineers for design roles due to their superior grasp on linear algebra and statistics. I'm paraphrasing but "it's easier to teach an IE how to do FEA than it is to teach a mechanical engineer DOE and Weibull analysis".
Why can't professors set LA as a prereq for their courses?
Or use it in their courses and earn students that they need to learn it so succeed?
It's up to department heads and dean and they're pressured by ABET and companies that write them checks.
Education is secondary; this is job training! We need to crank out people ready to drop into Boeing's way of doing things!
Yeah, stats is the other major deficiency in the course load. I think one course is required but its basically high school level "check out these normal distributions, 68-95-99.7, here's a Z-score, see you later".
Thankfully the companies I've worked for have done a really good job with advanced stats training.
What do these acronyms mean?—IE, FEA, DOE?
IE == Industrial Engineering: broad, but generally "Engineering of Systems" instead of a physical product. Laying out factories, setting up supply chains, etc. It's morphed a little bit from the original field so the name isn't super accurate.
FEA == Finite Element Analysis: advanced method of predicting the strength of a product via numerical simulation.
DOE == Design of Experiments: evaluation of how the outputs of a system change as you vary the inputs. At a high level, you build model of the system, then vary all the inputs through their entire range and to build a response surface of the output.
> It's morphed a little bit from the original field so the name isn't super accurate.
I'm currently doing a masters in IMSE--Industrial and Management Systems Engineering--and yeah it's changed since the 70s or whenever it had its real heyday (they come in waves).
The updated curriculum for undergrad is essentially the same as a mechanical engineering for the first two years, but as they wander off into advanced mechanics and fluids, IMSE students are doing time studies, factory design (FLOW!), and lots of stats and algorithms.
I've actually had the pleasure of getting to gripe to our school's Industrial Advisory Board which seems mostly full of Boeing people. They want to know if the curriculum serves the students well and I preach to them that, actually, if you spend 6 or 9 extra credits on proper software engineering that you've created a monster... but they don't listen. Some even get kind of offended because they think that a career in project management is a fine way to go about life (why go to engineering school?)
Sorry programming blows your mind? Perhaps that's why we need to teach it? I've done a lot of ERP integrations in my career and I'm not sure who they think is most qualified to do those sorts of things.
Currently watching:
https://ocw.mit.edu/courses/6-042j-mathematics-for-computer-...
and that assumption seems to be there as well, so very glad of the posting of the Youtube links elsethread.
Same... I didn't have to take Linear Algebra in ChemE undergrad. DiffEq had a little bit of LA... and ChemE had few classes where bits of pieces of LA where introduced and applied.
Graduate school definitely made up for lost time... LA was very front and center in the applied math courses.
Chapter 13 of the textbook was added in January 2022. It covers separating hyperplanes, signed distance to a hyperplane, Max-margin Classifiers, a remark on Soft Margin Classifiers, and the Orthogonal Projection Operator. The additional material was added to support EECS 445, Machine Learning at Michigan.
This is (one of?) the authors of the course, for what it's worth. Welcome to HN! Pelt him with questions, everybody. :)
For folks interested in 101 on linear algebra - I highly recommend book "Linear Algebra: Theory, Intuition, Code" by Mike X Cohen.
After trying a couple of courses and books, I liked it the most because it gives a pretty deep overview of the concepts, alongside the numpy and matlab code, which I found refreshing.
It's has good amount of proofs and has sections designed to build your intuition, which I really appreciated.
Chapter 13 of the textbook was added in January 2022. It covers separating hyperplanes, signed distance to a hyperplane, Max-margin Classifiers, a remark on Soft Margin Classifiers, and the Orthogonal Projection Operator. The material was added to support EECS 445, Machine Learning at Michigan.
MATH 214 (intro to Linear) was the least enjoyable class during my undergraduate at Umich. This seems like a better intro.
For engineering we had to pick either multivariate calculus or linear algebra for more upper level math courses. I picked multivariate, and I'll say it was also my least enjoyable there. I look back wondering what would have gone different if I picked linear algebra instead, but who knows, maybe I'd have just as blech of an experience with that. Lot of great classes in the EECS department though.
> Lot of great classes in the EECS department though.
Couldn’t agree more, Jack! Great times during 482… tranquil compared to the 470 slog that started immediately after every night :)
I totally remember 482 (Operating Systems for those reading) being really interesting. Story I remember is one of the final projects and dealing with locks in C++ world where I'd get close to full solution, but some errors from the locks, then I'd make a change and suddenly those previous failing tests passed but new ones failed. I didn't realize that could happen.
Great times. And I really liked how we did it all in C++ (other than computer vision 442 that was in matlab) rather than Python which some places do. Having that lower level understanding of languages in school makes understanding code so much easier, and something I didn't have to learn on my own.
MATH 217 was one of my favorites! Ive heard that the math department can be a little unenthusiastic about the non-major courses, but overall it’s a really welcoming place in my experience.
I love Linear Algebra. I took it in college almost 20 years ago and I still use it everyday. The higher level maths almost broke me academically. And it was a course in LA that really kept my head in the game. Even now, when I'm talking to students I try and encourage them to take the class if it's available.
For me LA was spread across several courses (I was at Michigan in engineering too), and I never got enough internalization of when it was useful from these, sadly. It definitely seemed more useful that a lot of the higher level maths, like you imply.
What's the best online credential for doing linear algebra? I like to do some self-studying but also, I'd like some form of "evidence" that I actually know my stuff and don't have t constantly explain that I do
I can't personally vouch for the program as I have not attended, but the University of Illinois offers quite a few mathematics courses online geared toward high school students, distance learners, and those preparing for grad school.
It is self-paced, so may not be what you're looking for, and it is expensive ($1250 if you have a BS already), but I seriously considered going this route before deciding to save big $$ and attend the local community college (which was actually a decent decision).
Program link: https://netmath.illinois.edu/
They offer 2 linear algebra courses, Math 257, which is Linear Algebra with Computer Applications (likely the "easy" applied version) and Math 416, Abstract Linear Algebra. Some of these Netmath courses do not have online lectures, but the Abstract LA course has video lectures from 2016.
From their site: "Math 416 is a rigorous, abstract treatment of linear algebra. Topics to be covered include vector spaces, linear transformations, eigenvalues and eigenvectors, diagonalizability, and inner product spaces. The course concludes with a brief introduction to the theory of canonical forms for matrices and linear transformations."
When I was investigating what to do in order to solidify my math credentials (still a work in progress), I knew UofI was a good school, and figured credit in one of their courses (online or not) would not be a terrible investment. At a bare minimum it wouldn't be belittled or untrusted like other online certificates might.
Plus the credit should transfer anywhere, if that's important.
> Math 416 is a rigorous, abstract treatment of linear algebra. Topics to be covered include vector spaces, linear transformations, eigenvalues and eigenvectors, diagonalizability, and inner product spaces. The course concludes with a brief introduction to the theory of canonical forms for matrices and linear transformations
Just be warned that this is literally the graduate level linear algebra course taken by mathematics majors. If you are looking for applications, this might not be it. On the other hand, if you are looking for a deep understanding of the fundamentals - I would say you found it.
Thank you for posting that info. I'd hate for the guy to want the applied computer-centric linear algebra only to find himself neck deep in a super rigorous course that might go deeper that he intended! Oof.
I should have posted the Math 257 description too. It also has lectures online as well as a synchronous Zoom component:
Introductory course incorporating linear algebra concepts with computational tools, with real world applications to science, engineering and data science. Topics include linear equations, matrix operations, vector spaces, linear transformations, eigenvalues, eigenvectors, inner products and norms, orthogonality, linear regression, equilibrium, linear dynamical systems and the singular value decomposition.
I'm from Europe, I suspect the credentials would still stand.
Who are you trying to prove this to?
Probably trying to transition from a SWE role to a ML one
I'd like to keep that option open yea, not sure if I want to. I just want to learn it, but I also want to prove to people that I can do it. This is one of the things I'm playing with.
Not why I'm going to study it though, but yea, I might want to switch.
If it's ML you want, you should check out Math Academy.
They have courses on linear algebra and mathematics for machine learning.
No certificates, but you can always demonstrate your mastery in interviews.
You can always build a project as well.
The writing is certainly all over the wall, in bold red ink.
I like Lay, it's one of the few math books anyone can read cover to cover, prove every statement and solving every problem, with no experience. It's like the Thomas' calculus equivalent linear algebra. If you do the work you'll get an easy A and will have built a great foundation for further engineering or theoretical study.
I taught numerical linear algebra in grad school and was really frustrated that even the applied math department took so long to build up to solving linear systems and eigen-decompsotions. The ordering of the material in the textbook is great, focusing on algorithms and decompositions.