• 0 Posts
  • 150 Comments
Joined 1 year ago
cake
Cake day: August 8th, 2023

help-circle










  • My Thinkpad almost turns 10 this year and I still use it. It’s still quite snappy for normal browsing and programming work on the go. Because I had 2 batteries for it that were easily switchable, the battery that I’m using now is not yet completely dead and will take me 1-2 hours of programming.

    It has had its screen replaced due to someone kicking my bag and breaking the screen, and I’ve had to replace the keyboard at one point after showering the laptop in tea. But the ease with which you can replace stuff like the keyboard is awesome. The thing is definitely built to last



  • Machine learning and compression have always been closely tied together. It’s trying to learn the “rules” that describe the data rather than memorizing all the data.

    I remember implementing a paper older than me in our “Information Theory” course at university that treated the creation of a decision tree as compression. Their algorithm considered sending the decisions tree and all the exceptions to the decision tree and the tree itself. If a node in the tree increased the overall message size, it would simply be pruned. This way they ensured that you wouldn’t make conclusions while having very little data and would only add the big patterns in the data.

    Fundamentally it is just compression, it’s just a way better method of compression than all the models that we had before.

    EDIT: The paper I’m talking about is “Inferring decision trees using the minimum description length principle” - L. Ross Quinlan & Ronald L. Rivest




  • I might misunderstand what you mean with “implementing” an LLM, but unless you have a good understanding of deep learning and math I wouldn’t recommend to implement one from scratch. There’s a lot of complex math involved in these kind of topics. If you mean implementing an application around an existing LLM, for example writing a chat website that interfaces with ChatGPT or a local LLM, then it’s doable (depending on you current skills).