I started off the quarter enrolled in this
statistical learning class, where they actually meant it when they said "strong background in linear algebra and statistics required." It literally killed me; hit me like a big, yellow school bus -- partly because the class was in the ECE department (and they take their math a lot more seriously than we do), and partly because, for some reason, I fail to be proficient at any kind of math learned after the eighth grade. After two painful iterations of problem sets and then handing in a practically blank midterm, my advisor and I agreed that it was probably best to drop the class and spend the time filling in the holes of my not-so-glamorous math career instead: a big relief, but also a huge disappointment at the same time.
However, the class did introduce me to what a deeper understanding of linear algebra should look like. Contrary to what I was taught in undergrad, linear algebra is more than just a nifty way to solve a system of linear equations that you otherwise could have done using techniques learned from middle-school algebra. It's even more than just learning the mechanics of matrix multiplication, eigenvector calculations, or following X number of steps to determine if some matrix is invertible. In fact, it seems as though I completely missed (or forgot?) the whole *point* of linear algebra in the first place, and am lacking a huge chunk of geometric intuition as a result.
I see more clearly now that (one of?) the over-arching purpose of linear algebra is to provide a means for elegantly organizing and operating on numbers in higher dimensions. Throughout undergrad, we were always used to working with scalars: x or y or z, with each variable representing a single number along the 1-dimensional number line. We could intuitively do all sorts of operations on these numbers: add/multiply, solve for unknowns, take derivatives/integrals, etc. But when we begin considering points in higher-dimensions, these methods break down because a point can longer be represented by a single number. Even in our friendly 2-dimensional space, with x and y axes, we now need two numbers, an x and y "coordinate", to identify a particular point in this space.
The use of linear algebra allows us to easily extend our ability to work in 1-dimensional space, all the way to n-dimensional space, even when we are not physically capable of visualizing these structures in high dimensions. It's a pretty simple (and perhaps obvious) idea to grasp, but it was a subtle revelation to me as I have been observing how the tools of linear algebra come into play in the larger framework of machine learning concepts and algorithms for high-dimensionality reduction (which is what my advisor specializes in -- more on that later). It's pretty mind-blowing, and I look forward to the day when I can just "see" these glorious planes and spaces running through what appears to be sets of seemingly unrelated numbers, filed neatly into rows and columns.
Until then, I will be slaving away at these
linear algebra lectures, re-learning undergrad-level linear algebra, and trying to figure out why, after all those years of asian-nerdiness-extra-math-homework-rampage, I still pee my pants everytime I see an integral. My bitterness toward the leniency of math requirements in computer science curriculums at american colleges can be saved for another day, but most importantly, the take-home message is this, kids: eat your wheaties and learn your linear algebra, and all will be well in machine learning land.
That, my friend, is a line from The Office.
One fortuitous afternoon, a friend sent me this link. It began with just a couple of episodes per week, during a late lunch or casual dinner. Then, it became a source of procastination ("Hm, what is more interesting than algorithms homework?"). Before I knew it, it became the solution to my every problem:
"I don't know why PCA has a lower error rate than LLE."
