TechTakes

I feel like the current Machine Learning gold rush is amazing from a technical perspective, but I feel like a lot of technophiles miss the real potential.

What we have is the first crickety engines of this technology. We're not building futurism masterpieces with the equivalent to a steam engine.

We have great new tools that we can use to further understand and optimize what we built, instead of just throwing more and more compute on top of our first design.

The human brain uses a light bulb of power, so we know we are massively inefficient. And recent research like MAMBA shows that there's still more improvements to make.

And Anthropic's Mech Interp shows there's ways to better understand the neural networks to improve performance without relying solely on a "black box".

The tech has great potential, but the massive server farms being dedicated to it now is just crypto style overhype and fear of missing out.