The question is: Is it like jpeg compression where the errors do not accumulate but the image comverges to a self inverse compressed image or does the data set converge to a single point which is meaningless?
The transformation function in jpeg (DCT) is generally well defined math. While lossy, most of the information is reprocudable.
An LLM is layers and layers of non-linear transformations. It's hard to say exactly how information is accumulated. You can inspect activations from tokens but it's really not clear how to define what the function is exactly doing. Therefore error is poorly understood.
JPEG is similar actually. The DCT is invertible, but the result of the DCT is quantized, which is where some of the compression happens (DCT -> quantization -> IDCT), so the end to end process is not truly invertible. Maybe an analogy to the non-linearities in between the linear steps in deep learning
just tried this:
take the graph of the functions x^n and exp(x)
how many points of intersection do they have?
chatgpt gave me the wrong answer, it claimed 2 points of intersection, but for n=4 there are 3 as one can easily derive. one for negative x and 2 points for positive x because exp(x) is growing faster than x^4.
then i corrected it and said 3 points of intersection. it said yes and gaev me the 3 points. then i said no there are 4 points of intersection and it again explained to me that there are 2 points of intersection. which is wrong.
then i asked it how many points of intersection for n=e and it said: zero
That not only works for compilation but also for more general code transformations like code parallelization with OpenMP in C and Fortran, Array-of-lists to List-of-arrays, or transform python code to parallel C-code and make a python module out of it.
I have created some pipelines this way where the LLM generates input files for a molecular dynamics code and write a python script for execution on a HPC system.
As far as I can tell I don’t think it is correct to say that this isn’t a matrix. B is just written down in transposed form. Whether that makes the math more or less clear is something you can argue for or against, but it’s the same math and it is confusing to call it something else.
It is a tensor of rank two with a special binary operation on tensors. These objects aren't matrices in the mathematical sense any more than convolution kernels aren't.
Matrices come with the matrix product defined over them.
This is one of four possible closed first order tensor contractions for an order two tensor, viz. AijBik AijBki AijBjk AijBkj. Only the third is applicable to matrices, all other contractions only work for general tensors without transposition.
What we deal with in computer science are actually n dimensional arrays since we don't have the co and contravariant indices that define tensors in physics.
The thing described by the article can be summarized by: "Any time you see A x B, you replace it with A x B^T" and it would, in practice, be exactly the same. I'm not sure the author understood this because then then go on to do a bunch of performance checks to see if there is any difference between the two. Which there isn't, because under the hood it is the exact same operations. They just multiple columns into columns (or rows into rows) instead of rows into columns. But the implicit transpose would undo that.
You can note (correctly) that this doesn't line up with the precise, but arbitrary traditional definition of a matrix, and that is correct. But that is just word games because you can very simply, using only syntax and no calculations, transform one into the other.
Damn, he mentioned Andy's Krabblergarten. Now it will be overrun with even more tourists.
My serious advice: Don't go there!! The Schnitzels are aweful and the Beergarden is ugly. ;-)
Too bad that Google, Apple, Intel now have offices here. This drives the rents up. Gentrification already killed the gay quarter. All luxury appartments and people now start complaining that the vibes of the quarter are gone. Who would have thought...
Not to worry, Andy's takes cash only, so probably 90% of readers here will have a bad experience ;) /s
it is one of the last cool places in that area, sadly it has been gentrified over the last 20 years.
Haxnbauer, however, has and always will be a tourist trap, even now that there are 2 of them. New haxenbauer in the original haxenbauer place and original haxenbauer in a new place, about 200m further.
For anyone else wanting a good experience and good food (and being able to pay with card) i always recommend:
Hamburg is more conservative than Munich (I have lived in both cities). It is more like Austin wehen you compare it to a city in Texas. And Hamburg is in no way as hippy as San Francisco was. The closest I would compare Hamburg to is Boston.
In the golden 70s Munich was a melting pot for musicians, gay people, Hippies. They still have the nudist beach in the city centre. Try to find something like that in the US.
Now compared to Berlin I think Hamburg is still pretty conservative and I am not in a position to make apt comparisons to cities in the US but I do have to disagree with the statement that Hamburg is more conservative.
I was born and raised in Hamburg and lived there or in adjacent parts most of my life. I also visited Munich quite a few times due to a long distance relationship and I would disagree that Hamburg is more conservative. tThe people in Munich vote for conservative parties at a greater rate than the people in Hamburg and Munich never felt even remotely as multi-cultural as Hamburg. I distinctly remember walking around München for the first time and being surprised by people’s reactions to seeing a black guy walking down the street. Some people would literally stop walking and stare. Almost no Middle Eastern people either in comparison.
There is also a pretty strong divide between the north being much less religious. And one might argue that the people who are Christians are more often Protestant in the north which is arguably more progressive than the catholics in the south. If you look at Hamburg during may 1st, consider the Rote Flora building and the Schanzenviertel I think it’s quite clear that Hamburg has a pretty firmly established left-wing community. Granted if you go to Blankenese or the Neue Hafencity (areas for and of the wealthy) and talk to the people living there you might get a different picture. Anyways talking in averages I am not convinced your statement holds true today.
I think there is sort of a cultural rivalry where people from the north don’t want to get confused with the people from the south of Germany and vice versa. We make fun of their way they butcher the language and their festivities and traditional attire, and how they talk too much, and they make fun of us for being tight lipped humorless pricks.
Still in the 70s? I thought the last time that Munich had bohemians was between the two world wars. Hitler was one of them. Yes, really. He formed his ideology in that environment.
Hamburg has a harbor and is at the riverside of a large stream so that makes it a complete different setup. Munich on the other hand still has a castle in the inner city (Residenz) and has many historical buildings. Hamburg has the status of a "state" in Germany while Munich is the capital of Bavaria (a state in Germany). Both cities are rich and the rich people also celebrate the weath in both cities. If you want a party city you better go to Berlin. (Arm aber sexy (Poor but sexy)) is their slogan.
Wasn't much of the historically significant architecture in Munich destroyed? How historically significant are restored buildings on sites that were razed or heavily damaged?
Feels like going to a natural history museum and just seeing reproductions. Educational but not "real"
Well I live very close to Theresienwiese and dont have any problems at all. During the 2 weeks of Octoberfest they clean the streets every morning at 4am and everything is clean and shiny again. Also now that the Theresienwiese have been encircled by a fence during Octoberfest the numbers of drunk people has fallen dramatically.
Munich has a smaller version of the Octoberfest all year around. Beer Gardens everywhere, many people wearing "Tracht". The tourists love it and the munich people love the money. It has some nice places like the English Garden or the River Side. Also the Farmers Market in the city centre is nice to hang around. It is not for the young crowd but more for the settled and wealthy ones. Very much like Zürich, but bigger. The mountains are close and the lakes around Munich are famous. Also many nice castles like Schloss Nymphenburg, the Residenz and of course the castle built by Ludwig II. Lots of historic buildings. The city was founded 1158 and parts of the old city still exist.
Münchner here. It is a very touristy city. Dirty? not so much in comparison to other german cities like Cologne or Berlin.
Uninviting and boring depends what you want to do. Nightlife in comparison to Berlin is poor. All the techno clubs are there. Octoberfest is a melting pot for the whole world. Americans and Australiens have a hard time digesting the amounts of beer.
