Collecting Data Is Hard

  • Comments posted to this topic are about the item Collecting Data Is Hard

  • Human brains as spaceship controllers is not a new concept: The ship who sang is almost 60 years old now.

  • DISCLAIMER:  The following response was AI-generated as a way to help make my point!!!!


    That anxiety about the "synthetic data feedback loop" is completely valid, as model collapse from consuming AI-generated content is becoming a real technical bottleneck. When human-driven communities like Stack Overflow or SQL Server Central lose active contributors, we lose the messy, edge-case nuance that synthetic data completely fails to replicate. If Garbage In, Garbage Out was a headache with legacy databases, watching AI models train on recycled "AI slop" is going to make system performance downright unpredictable in the real world.


    We now return you to an actual human being writing a response, thank you for your attention.

    So, yes, I did copy/paste the editorial into a LLM and asked it to generate a response.  Frankly, I feel it does point out the problem the editorial, you get a response that sounds reasonable, reads a bit stilted, but took very little effort on my part.  When a techie site such as SSC or Stack Overflow starts getting more replies to questions that are generated by AI compared to a real person, people start leaving.

    Why?  Because yes, you're dealing with GIGO and what's the saying?  99% of everything is crap?  And the LLMs were trained largely ON that crap.  So you get "Jimmy Look At How Many Questions I Answer" who's just feeding the question into an LLM, grabbing the "answer," and pasting it in.  But the "answer" is not a GOOD answer or even is outright WRONG, which annoys the users of the site that spent time making sure their answer was both correct and good, and perhaps they don't even bother responding.

    Or the posts I see on other sites that all have the same format, the "I was annoyed at X with service Y, so I built Z..." which just strike me as a case of the poster going to their LLM of choice and asking it "oh great LLM of the lake, write me an application to do Z and then a post for site W telling them what I did."  And, I ignore and skim past those posts.

    The problem is, some people (I do it sometimes, I know) write in the same manner that responses from the LLMs sound, so filtering out ACTUAL LLM posts from a real persons' posts, is not exactly an easy (perhaps not even possible) task.  Honestly, I don't know what might be a solution...

  • I don't either. I use a service to check submissions for articles for LLM type things. There are some signs, but sometimes it's hard.

    And sometimes the LLM stuff reads pretty well. There are good outputs from LLMs, especially with some guidance. I think I often throw away 76-90% of what an LLM gives me and edit, but it's still useful. It gives me something to throw away. At best, I think I've kept about 50% of the LLM stuff and then worked around it.

    The separate thing is that the body of knowledge isn't necessarily growing in creative ways. Maybe lightly, but a) is that good enough to continue to advance and b) does it matter is most people use LLMs.

    I have no idea here.

  • Good point that people seem to be sharing a lot less than they did before. I've seen this in a Slack group I'm in for Azure DevOps and GitHub. There are a couple thousand members of this Slack group. It used to be very busy until 18 months ago. I thought that something had gone wrong or, well something had failed. I asked a few guys I'm friends with to find out why the chatter has gone to zero. They said exactly what you said. People are using various LLMs to answer their questions and no longer share with others what they've learned. That is very sad. I just didn't realize how widespread this is getting to be!

    Kindest Regards, Rod Connect with me on LinkedIn.

  • Wasnt that the gist of Firefox? (I know, I should have actually read it!) but it was supposed to be an fighter jet that you controlled with your mind... and that was like 50 years ago.

Viewing 6 posts - 1 through 6 (of 6 total)

You must be logged in to reply to this topic. Login to reply