Linear Algebra Done Right – 4th Edition

> This is actually why I feel that mathematical texts tend to be not rigorous enough, rather than too rigorous.

The thing that mathematicians refuse to admit is that they are extremely sloppy with their notation, terminology and rigor. Especially in comparison to the average programmer.

They are conceptually/abstractly rigorous, but in "implementation" are incredibly sloppy. But they've been in that world so long they can't really see it / just expect it.

And if you debate with one long enough, they'll eventually concede and say something along the lines of "well math evolved being written on paper and conciseness was important so that took priority over those other concerns." And it leaks through into math instruction and general math text writing.

Programming is forced to be extremely rigorous at the implementation level simply because what is written must be executed. Now engineering abstraction is extremely conceptually sloppy and if it works it's often deemed "good enough". And math generally is the exact opposite. Even for a simple case, take the number of symbols that have context sensitive meanings and mathematicians. They will use them without declaring which context they are using, and a reader is simply supposed to infer correctly. It's actually somewhat funny because it's not at all how they see themselves.

> Remark. It is easy to prove that zero vector 0 is unique, and that given v ∈ V its additive inverse −v is also unique.

I'm sorry, this book is meant for the audience who can read and write proofs. Uniqueness proofs are staple of mathematics. If word "unique" throws you off, then this book is not meant for you.

Mathematicians are well aware of complaints like these about introductions to their subjects, by the way.

It is for a reason that this book introduces the theory of abstract vector spaces and linear transformations, rather than relying on the crutch of intuition from Euclidean space. If you want to become a serious mathematician (and this is a book for such people, not for people looking for a gentle introduction to linear algebra for the purposes of applications) at some point it is necessary to rip the bandaid of unabstracted thinking off and engage seriously with abstraction as a tool.

It is an important and powerful skill to be presented with an abstract definition, only loosely related to concrete structures you have seen before, and work with it. In mathematics this begins with linear algebra, and then with abstract algebra, real analysis and topology, and eventually more advanced subjects like differential geometry.

It's difficult to explain to someone whose exposure to serious mathematics is mostly on the periphery that being exposed forcefully to this kind of thinking is a critical step to be able to make great leaps forward in the future. Brilliant developments of mathematics like, for example, the realisation that "space" is an intrinsic concept and geometry may be done without reference to an ambient Euclidean space begin with learning this kind of abstract thinking. It is easy to take for granted the fruits of this abstraction now, after the hard work has already been put in by others to develop it, and think that the best way to learn it is to return back to the concrete and avoid the abstract.

Axler serves as an adequate first introduction to linear algebra (though it is intended to be a second, more formal, pass through. Think analysis vs calculus), but it isn't intended to be a first introduction to all of formal mathematics! A necessary prereq is understanding some formal language used in mathematics- what unique means is included in that.

Falling entirely back on physical intuition is fine for students who will use linear algebra only in physical contexts, but linear algebra is often a stepping stone towards more general abstract algebra. That's what Axler aims to help with, and with arbitrary (for instance) rings there isn't a nice spacial metaphor to help you. There you need to have developed the skill of looking at a definition and parsing out what an object is from that.

> This is actually why I feel that mathematical texts tend to be not rigorous enough, rather than too rigorous.

This is _precisely_ the opinion of Roger Godement, French mathematician and member of the Bourbaky group.

I would highly recommend his books on Algebra. They are absolutely uncompromising on precision and correctness, while also being intuitive and laying down all the logical foundations of their rigor.

Overall, I cannot recommend enough the books of the Bourbaky group (esp. Dieudonne & Godement). They are a work of art in the same sense that TAOCP is for computer science.

I absolutely agree about additional rigor and precision making math easier to learn. Only after you're familiar with the concepts can you be more lazy.

That's the approach taken by my favorite math book:

https://people.math.harvard.edu/~shlomo/docs/Advanced_Calcul...

> This is actually why I feel that mathematical texts tend to be not rigorous enough, rather than too rigorous. On the surface the opposite is true - you complain, for instance, that the text jumps immediately into using technical language without any prior introduction or intuition building. My take is that intuition building doesn't need to replace or preface the use of formal precision, but that what is needed is to bridge concepts the student already understands and has intuition for to the new concept that the student is to learn.

If you read the book in the original post you may find it's absolutely for you.

Axler assumes you know only the real numbers, then starts by introducing the commutative and associative properties and the additive and multiplicative identity of the complex numbers[1]. Then he introduces fields and shows that, hey look we have already proved that the real and complex numbers are fields because we've established exactly the properties required. Then he goes on to multidimensional fields and proves the same properties (commutativity and associativity and identities) in F^n where F is any arbitrary field, so could be either the real or the complex numbers.

Then he moves onto vectors and then onto linear maps. It's literally chapter 3 before you see [ ] notation or anything that looks like a matrix, and he introduces the concept of matrices formally in terms of the concepts he has built up piece by piece before.

Axler really does a great job (imo) of this kind of bridge building, and it is absolutely rigorous each step of the way. As an example, he (famously) doesn't introduce determinants until the last chapter because he feels they are counterintuitive for most people and you need most of the foundation of linear algebra to understand them properly. So he builds up all of linear algebra fully rigorously without determinants first and then introduces them at the end.

[1] eg he proves that there is only one zero and one "one" such that A = 1*A and A = 0 + A.

I think this followed or accompanied by axler is the way to go

This, betterexplained, ritvikmath, SeeingTheory will give you a very solid math background(I think they are better than 90% of the intro math classes in colleges).

> No

Gilbert Strang (already mentioned by fellow commenters).

> The subject is too young

"The first modern and more precise definition of a vector space was introduced by Peano in 1888; by 1900, a theory of linear transformations of finite-dimensional vector spaces had emerged." (from Wikipedia)

What? Linear Algebra is easily one of the best understood fields of mathematics. Maybe elementary number theory has it beat, but the concepts that drive useful higher level number theory aren't nearly so clear or direct as those driving linear algebra. It's used as a lingua franca between all sorts of different subjects because mathematicians of all stripes share an understanding of what it's about.

From what you said there, it seems like you tried to approach linear algebra from nearly random directions- and often from the end rather than the beginning. If you're in it for the computation, Axler definitely isn't for you. There are texts specifically on numeric programming- they'll jump straight to the real world use. If you want to understand it from a pure math perspective, I'd recommend taking a step back and tackle a textbook of your choosing in order. The definition of a dual space makes a lot more sense once you have a vector space down.

Why should it buy you something is the real question.

You don't need to understand it the way the "initial" author thought about it, should that person had given it more thoughts...

History of maths is really interesting but it's not to be confused with math.

Concepts are not useful as you think about them in economic opportunity case. Think about them as "did you notice that property" and then you start doing math, by playing with these concepts.

Otherwise you'll be tied to someones way of thinking instead of hacking into it.

> Why should I care about different forms of matrix decomposition? What do they buy me?

A natural line of questioning to go down once you're acquainted with linear maps/matrices is "which functions are linear"/"what sorts of things are linear functions capable of doing?"

It's easy to show dot products are linear, and not too hard to show (in finite dimensions) that all linear functions that output a scalar are dot products. And these things form a vector space themselves, the "dual space" (because each element is a dot-product mirror of some vector from the original space). So linear functions from F^n -> F^1 are easy enough to understand.

What about F^n -> F^m? There's rotations, scaling, projections, permutations of the basis, etc. What else is possible?

A structure/decomposition theorem tells you what is possible. For example, the Jordan Canonical Form tells you that with the right choice of basis (i.e. coordinates), matrices all look like a group of independent "blocks" of fairly simple upper triangle matrices that operate on their own subspaces. Polar decomposition says that just like complex numbers can be written in polar form re^it, where multiplication scales by r and rotates by t, so can linear maps be written as a higher dimensional multiplication/scaling and orthogonal transformation/"rotation". The SVD says that given the correct choice of basis for the source and image, linear maps all look like multiplication on independent subspaces. The coordinate change for SVD is orthogonal, so another interpretation is that roughly speaking, SVD says all linear maps are a rotation, scaling, and another rotation. The singular vectors tell you how space rotates and the singular values tell you how it stretches.

So the name of the game becomes to figure out how to pick good coordinates and track coordinate changes, and once you do this, linear maps become relatively easy to understand.

Dual spaces come up as a technical thing when solving PDEs for example. You look for "distributional" solutions, which are dual vectors (considering some vector space of functions). In that context people talk about "integrating a distribution with test functions", which is the same thing as saying distributions are dot products (integration defines a dot product) aka dual vectors. There's some technical difficulties here though because now space is infinite dimensional, and not all dual vectors are dot products, e.g. the Dirac delta distribution delta(f) = f(0) can't be written as a dot product <g,f> for any g, but it is a limit of dot products (e.g. with taller/thinner gaussians). One might ask whether all dual vectors are limits of dot products and whether all limits of dual vectors are dual vectors (as limits are important when solving differential equations). The dual space concept helps you phrase your questions.

They also come up a lot in differential geometry. The fundamental theorem of calculus/Stokes theorem more-or-less says that differentiation is the adjoint/dual to the map that sends a space to its boundary. I don't know off the top of my head of more "elementary" examples. It's been like 10 years since I've thought about "real" engineering, but roughly speaking, dual vectors model measurements of linear systems, so one might be interested in studying the space of possible systems (which, as in the previous paragraph, might satisfy some linear differential equations). My understanding is that quantum physics uses a dual space as the state space and the second dual as the space of measurements, which again seems like a fairly technical point that you get into with infinite dimensions.

Note that there's another factoring theorem called the first isomorphism theorem that applies to a variety of structures (e.g. sets, vector spaces, groups, rings, modules) that says that structure-preserving functions can be factored into a quotient (a sort of projection) followed by an isomorphism followed by an injection. The quotient and injection are boring; they just collapse your kernel to zero without changing anything else, and embed your image into a larger space. So the interesting things to study to "understand" linear maps are isomorphisms, i.e. invertible (square) matrices. Another way to say this is that every rectangular matrix has a square matrix at its heart that's the real meat.

From "Preface to Students":

> You are probably about to begin your second exposure to linear algebra. Unlike your first brush with the subject, which probably emphasized Euclidean spaces and matrices, this encounter will focus on abstract vector spaces and linear maps. These terms will be defined later, so don’t worry if you do not know what they mean. This book starts from the beginning of the subject, assuming no knowledge of linear algebra. The key point is that you are about to immerse yourself in serious mathematics, with an emphasis on attaining a deep understanding of the definitions, theorems, and proofs.

It is definitely a hard text if you haven't had exposure to linear algebra before.

Those two are pretty big as far as specific examples go, definitely worth writing about

I care not for this explanation and opinion. If someone is unable to work on practice problems without just looking up the solutions that is really their problem, not mine.

> And you’re supposed to have teaching assistants or similar available when you really remain stuck.

That just translates to giving the middle finger to self learners.

I have not yet seen a decent explanation for withholding sample solutions/explanations. It usually just boils down to "I want this to be only for university professors to give homework from and I am of the opinion that university students lack the minimal discipline to work on problems in their own"

Yeah sorry that's just total and utter BS. No sale.

I am an adult. >99% of students of university subjects are adults. Even assuming your premise totally, anyone so incredibly stupid they can't have it explained in the text when is the right time to use the solutions is too stupid to learn from the text. Axler puts them on a website that is less than 30 seconds away.

"No choice..." Describes what proportion of texbook sales?

That's entirely possible, but in the context of introductory books I think it's fair to assume & limit scope to Strang's "Introduction to Linear Algebra" and Axler's "Linear Algebra Done Right."

I am an applications-oriented person and my inclination was to go directly from a matrix/determinant heavy picture into applications. Strang['s intro text] only. I am extremely glad that someone intercepted me and made me get some practice with abstract vector spaces, operators, and inner product spaces first, using Axler. This practice bailed me out and differentiated me from peers on a number of occasions, so I want to pass down the recommendation.

Perhaps, but that's about as useful as pointing out that monads are a monoid in the category of endofunctors. What's the "image of a matrix?" Coming at LA from a 3D graphics background, I've never heard that term before. And what does the "span of its columns" mean?

To me, each column represents a different dimension of the basis vector space, so the notion that X, Y, and Z might form independent "column spaces" of their own is unintuitive at best.

These are all questions that can be Googled, of course, but in the context of a coherent, progressive pedagogical approach, they shouldn't need to be asked. And they certainly don't belong in the first chapter of any introductory linear algebra text, much less the preface.

I'm not saying they don't belong in the book. I'm saying, evidently poorly, that they don't belong in the first chapter.

Strang has a bit of a "Next, draw the rest of the fucking owl" vibe going on. It wasn't what I'd been led to expect from the reviews.

>>The applications are endless!

Pretty much all of grad level engineering classes, all of grad level Stats classes, all of grad level Econ classes include applications of LA. Heck quantitative research work in social sciences are still included via stats.

Same time. I'd taught myself linear algebra from the Strang lectures (and a Slack study group we set up with some random university's syllabus, which gave us a set of homework problems to do) long before this, so mostly I just matched the professor's lectures to the Strang material, and dipped in and out of Axler when proof and conceptual stuff came up; it's not like we did Axler cover to cover.

Before doing this, I'd only ever sort of skimmed Axler; it's sort of not the linear algebra you care about for cryptography, and up until the spectral theorem stuff that's exactly what Strang was. It was neat to get an appreciation for Axler this was.

That's awesome. I bet TAing for Strang was a great experience. Now I'm curious if you have any recommendations for "actually calculating something"