Formerly u/CanadaPlus101 on Reddit.

  • 3 Posts
  • 472 Comments
Joined 1 year ago
cake
Cake day: June 12th, 2023

help-circle




  • Maybe I just like the idea of a closing tag being very specific about what it is that is being closed (?).

    That’s kind of what I was getting at with the mental scoping.

    My peeve with json is that… it doesn’t properly distinguish between strings that happen to be a number and “numbers"

    Is that implementation-specific, or did they bake JavaScript type awfulness into the standard? Or are numbers even supported - it’s all binary at the machine level, so I could see an argument that every (tree) node value should be a string, and actual types should be left to higher levels of abstraction.

    I actually don’t like the attributes in xml, I think it would be better if it was mandatory that they were also just more tagged elements inside the others, and that the “validity” of a piece of xml being a certain object would depend entirely on parsing correctly or not.

    I particularly hate the idea of attributes in svg, and even more particularly the way they defined paths.

    I agree. The latter isn’t even a matter of taste, they’re just implementing their own homebrew syntax inside an attribute, circumventing the actual format, WTF.



  • I think we did a thread about XML before, but I have more questions. What exactly do you mean by “anything can be a tag”?

    It seems to me that this:

    <address>
        <street_address>21 2nd Street</street_address>
        <city>New York</city> 
        <state>NY</state>
        <postal_code>10021-3100</postal_code>
    </address>
    

    Is pretty much the same as this:

      "address": {
        "street_address": "21 2nd Street",
        "city": "New York",
        "state": "NY",
        "postal_code": "10021-3100"
      },
    

    If it branches really quickly the XML style is easier to mentally scope than brackets, though, I’ll give it that.














  • Yep.

    I’d still call that memory. It’s not the present; arguably for a (post-training) LLM the present totally consists of choosing probabilities for the next token, and there is no notion of future. That’s really just a choice of interpretation, though.

    During training they definitely can learn and remember things (or at least “learn” and “remember”). Sometimes despite our best efforts, because we don’t really want them to know a real, non-celebrity person’s information. Training ends before the consumer uses the thing though, and it’s kind of like we’re running a coma patient after that.