PixelProf @ PixelProf @lemmy.ca

Posts

3

Comments

144

Joined

2 yr. ago

Hmm... Nothing off the top of my head right now. I checked out the Wikipedia page for Deep Learning and it's not bad, but quite a bit of technical info and jumping around the timeline, though it does go all the way back to the 1920's with it's history as jumping off points. Most of what I know came from grad school and having researched creative AI around 2015-2019, and being a bit obsessed with it growing up before and during my undergrad.

If I were to pitch some key notes, the page details lots of the cool networks that dominated in the 60's-2000's, but it's worth noting that there were lots of competing models besides neural nets at the time. Then 2011, two things happened at right about the same time: The ReLU (a simple way to help preserve data through many layers, increasing complexity) which, while established in the 60's, only swept everything for deep learning in 2011, and majorly, Nvidia's cheap graphics cards with parallel processing and CUDA that were found to majorly boost efficiency of running networks.

I found a few links with some cool perspectives: Nvidia post with some technical details

Solid and simplified timeline with lots of great details

It does exclude a few of the big popular culture events, like Watson on Jeopardy in 2011. To me it's fascinating because Watson's architecture was an absolute mess by today's standards, over 100 different algorithms working in conjunction, mixing tons of techniques together to get a pretty specifically tuned question and answer machine. It took 2880 CPU cores to run, and it could win about 70% of the time at Jeopardy. Compare that to today's GPT, which while ChatGPT requires way more massive amounts of processing power to run, have an otherwise elegant structure and I can run awfully competent ones on a $400 graphics card. I was actually in a gap year waiting to go to my undergrad to study AI and robotics during the Watson craze, so seeing it and then seeing the 2012 big bang was wild.

Yeah I probably should have added the /s to that one.

Oh for sure. And it's a great realm to research, but pretty dirty to rip apart another field to bolster your own. Then again, string theorist...

For me, it's the next major milestone in what's been a roughly decade-ish trend of research, and the groundbreaking part is how rapidly it accelerated. We saw a similar boom in 2012-2018, and now it's just accelerating.

Before 2011/2012, if your network was too deep, too many layers, it would just breakdown and give pretty random results - it couldn't learn - so they had to perform relatively simple tasks. Then a few techniques were developed that enabled deep learning, the ability to really stretch the amount of patterns a network could learn if given enough data. Suddenly, things that were jokes in computer science became reality. The move from deep networks to 95% image recognition ability, for example, took about 1 years to halve the error rate, about 5 years to go from about 35-40% incorrect classification to 5%. That's the same stuff that powered all the hype around AI beating Go champions and professional Starcraft players.

The Transformer (the T in GPT) came out in 2017, around the peak of the deep learning boom. In 2 years, GPT-2 was released, and while it's funny to look back on now, it practically revolutionized temporal data coherence and showed that throwing lots of data at this architecture didn't break it, like previous ones had. Then they kept throwing more and more and more data, and it kept going and improving. With GPT-3 about a year later, like in 2012, we saw an immediate spike in previously impossible challenges being destroyed, and seemingly they haven't degraded with more data yet. While it's unsustainable, it's the same kind of puzzle piece that pushed deep learning into the forefront in 2012, and the same concepts are being applied to different domains like image generation, which has also seen massive boosts thanks in-part to the 2017 research.

Anyways, small rant, but yeah - it's hype lies in its historical context, for me. The chat bot is an incredible demonstration of the incredible underlying advancements to data processing that were made in the past decade, and if working out patterns from massive quantities of data is a pointless endeavour I have sad news for all folks with brains.

Windows 11 has tabbed file explorer, a package manager, it's quick, the interface looks nice and feels nice, and it's been really stable for me. I don't know where the complaints are at, it's been great. All they need to do is regress all of the ads-in-your-OS stuff from 10. Bring back the start menu that doesn't hang for 30 seconds looking something up online before showing you your installed programs.

I understand that he's placing these relative to quantum computing, and that he is specifically a scientist who is deeply invested in that realm, it just seems too reductionist from a software perspective, because ultimately yeah - we are indeed limited by the architecture of our physical computing paradigm, but that doesn't discount the incredible advancements we've made in the space.

Maybe I'm being too hyperbolic over this small article, but does this basically mean any advancements in CS research are basically just glorified (insert elementary mechanical thing here) because they use bits and von Neumann architecture?

I used to adore Kaku when I was young, but as I got into academics, saw how attached he was to string theory long after it's expiry date, and seeing how popular he got on pretty wild and speculative fiction, I struggle to take him too seriously in this realm.

My experience, which comes with years in labs working on creative computation, AI, and NLP, these large language models are impressive and revolutionary, but quite frankly, for dumb reasons. The transformer was a great advancement, but seemingly only if we piled obscene amounts of data on it, previously unspeculated of amounts. Now we can train smaller bots off of the data from these bigger ones, which is neat, but it's still that mass of data.

To the general public: Yes, LLMs are overblown. To someone who spent years researching creativity assistance AI and NLPs: These are freaking awesome, and I'm amazed at the capabilities we have now in creating code that can do qualitative analysis and natural language interfacing, but the model is unsustainable unless techniques like Orca come along and shrink down the data requirements. That said, I'm running pretty competent language and image models on 12GB of relatively cheap consumer video card, so we're progressing fast.

Edit to Add: And I do agree that we're going to see wild stuff with quantum computing one day, but that can't discount the excellent research being done by folks working with existing hardware, and it's upsetting to hear a scientist bawk at a field like that. And I recognize I led this by speaking down on string theory, but string theory pop science (including Dr. Kaku) caused havoc in people taking physics seriously.

I'm not sure if it's relevant here, but I'd recommend taking a look at the book Adult Children of Emotionally Immature Parents. I picked up the audiobook from my library and it really helped me understand myself, my development, and my parents a lot better and to have a healthier outlook on our relationship. I always understood my parents had their own baggage, but I didn't realize the specifics I could be on the lookout for, the specific reactions I'd had that could be linked to it, and how to move forward.

It could at least be a good start. Best of luck!

I really appreciate hearing this. I was diagnosed ADHD almost a year ago, and it made a lot of things make sense, but therapy continues coming back to, "Yes, definitely the ADHD, it's obvious now, but there's just something... else." I'm going to begin trauma therapy soon, but stumbling upon some information on autism made me immediately recoil and say "That can't be me". When I learned a bit more, I realized where the reaction came from. I've spent a lot of time and energy avoiding and pushing away those traits. I blocked out a lot of memories as a kid, and as soon as I read about common autistic behaviours in children I started to be able to see those things and remember them and not immediately recoil, but suddenly consider them as something that really happened.

I think that's the real fear of "high-functioning", whatever it is, without having a label/diagnosis. Yes, I'm participating in society, I'm doing well, people like me, I'm good at my job - I guess it's just that the cost of it is everything that makes me me, and if that's normal, I don't understand how anyone gets by - and if it isn't normal, then what is it so much harder for me, and what does it mean for me?

So yeah, I do think I might look into assessment. I think for me, I've just pushed myself so hard for so long and blamed myself for so much that painful perseverance is deeply ingrained. Even though I should be able to follow my needs and rhythms without a diagnosis, it's a lot easier to look at yourself when you can see the full picture and be properly honest.

Plus, masking is exhausting, even when it's subconscious. Learning about ADHD got me to learn some things I'd been painfully masking without being completely aware. There's a lot of programming in me that feels natural, but it's just a reaction to how things "should" be done that really don't need to be there.

So yeah, I really appreciate the comment, and I think I'll probably find a lot of commonalities in your journey :)

This is the start of the use cases I wanted to see take off with Mastodon/Lemmy/Kbin. Much like the previous era of distributed content with user-hosted voice servers and forums, having larger communities/organizations run their own instances and avoid trying to treat the space as one big pool of content is the real use case here. The fact that you can cross-instance subscribe and post makes it viable long-term.

It also gives "free" verification of information's sources based on the domain, the same way that (modern) email gives you an extra layer of confidence when you see a verified domain. I would love the see the Government of Canada, CBC, Universities, all starting their own instances and utilizing them in unique and interesting ways. With enough adoption, official provincial/municipality instances could pop up to make organized communities easier.

It feels to me like a starting move away from the autocracy that the platform economy has created. It's not universal, but I absolutely push back against too many instances trying to be "general purpose Reddit replacements" because that seems like a fleeting use case for what it can eventually become, and it just confuses the whole abstraction of what these decentralized socials afford.

Poorly.

More seriously, I didn't know I had ADHD, but I'd kind of naturally contorted my world to support it as best as I could. I worked flexible, four months contracts. I only worked in low-stakes positions where leaving after a few months was expected. When I was a young kid, I was really good at convincing teachers that they didn't need to see my homework or that I needed an extra day, because even though the work was trivial, I wouldn't do it until the day after a deadline.

I've minimized obligations where I can, like autopay for every bill, I don't drive to avoid having to take it to the shop and do maintenance, I rent so that I'm not on the hook for maintenance, and I chronically overthink purchases to avoid impulse spending most of the time, at the sacrifice of not getting things I probably need.

I'm still working on it, but I think reducing the places where you can really mess things up on a bad brain day and doing what you can to nurture an environment where you can follow your rhythms is important. Way easier said than done, of course.

Even thinking about ASD as a possibility for me is a very new thing, so I don't know with any certainty I fall here, but learning about "high-functioning" autism as being "successfully masked" has made me seriously begin exploring this. Psychiatrists were hesitant early on to entertain any discussions (edit: of any neurodivergence) because I'd been so successful in school and career, even though I was always dying inside feeling like an imposter in society and not knowing who I am and why every interaction feels so difficult and requires so much effort. Why does just existing feel so deeply uncomfortable?

I wrote off the idea of high-functioning ASD because I certainly didn't feel high-functioning, despite my successes, because of the feeling of claustrophobia in my own skin. I wrote off any other higher needs because I didn't have those needs and I'd exhausted a lot of time and effort to not need those supports... Which in hindsight, is probably not typical.

Neurodivergence is so often defined by the ability to fit in and avoid disrupting typical society and norms. I wasn't depressed enough because I went into work enough days. I wasn't ADHD enough because I wasn't disrupting other people's lives and breaking social rules and got good grades. I wasn't autistic enough because I learned to to navigate social situations (read: only engaged in social situations where I'm in a position of control or known mutual interest).

"High-functioning" for whom?

Anyways again, I'm new to even remotely considering autism for me, so I apologize if this is wholely unrelated and unrelatable, but whatever flavour my brain is, it tastes a lot worse than it smells (only figuratively... As far as I know).

Soap. 100%.

Yeah, I'm not personally attributing malice, I've just felt like it's a bit of a show of power/absurdity of wealth that he can kind of accidentally buy one of the largest companies, mess with it for the lulz without caring about the consequences. It seems like the stupidity I would have done of I was a billionaire when I was a nihilistic teen high on chans boards.

I've gotten by too long and too successfully letting my impulses keep everything chugging along well enough that now that there's a massive range of responsibility, dependence upon me, and deadlines with major consequences, hoping I'll impulsively get around to things had begun putting a painful spotlight on previously undiagnosed ADHD.

I think the key to learning is to not do the thing out of impulse, but to train setting a goal of doing the thing and then painstakingly doing nothing until that thing is done. That's the skill to train, not the thing that's getting done.

But now, I'll just wait for the time that that skill is the impulsive thing to work on and keep on keeping on.

I know this post and comment might sound shilly but switching to more expensive microfibre underwear actually made a big impact on my life and motivated me to start buying better fitting and better material clothes.

I'd always bought cheap and thought anything else was silly. I was wrong. So much more comfortable, I haven't had a single pair even begin to wear down a little bit, less sweating and feel cleaner, fit better, and haven't been scrunchy or uncomfortable once compared to the daily issues of that cheap FotL life. This led to more expensive and longer lasting socks with textures I like better, better fitting shoes that survive more than one season.

It was spawned by some severe weight loss and a need to restock my wardrobe. My old underwear stuck around as backups to tell me I needed to do laundry, but going back to the old ones was bad enough that I stopped postponing laundry.

Basically, I really didn't appreciate how much I absolutely hated so many textures I was constantly in contact with until I tried alternative underwear and realized you don't have to just deal with that all the time.

It depends what "From Scratch" means to you, as I don't know your level of programming or interests, because you could be talking about making a game from beginning to end, and you could be talking about...

• Using a general purpose game engine (Unity, Godot, Unreal) and pre-made assets (e.g., Unity Asset Store, Epic Marketplace)?
• Using a general purpose game engine almost purely as a rendering+input engine with a nice user interface and building your own engine overtop of that
• Using frameworks for user input and rendering images, but not necessarily ones built for games, so they're more general purpose and you'll need to write a lot of game code to put it all together into your own engine before you even starting "Making the game", but offer extreme control over every piece so that you can make something very strange and experimental, but lots of technical overhead before you get started
• Writing your own frameworks for handling user input and rendering images... that same as previous, but you'll spend 99% of your time trying to rewrite the wheel and get it to go as fast as any off the shelf replacement

If you're new to programming and just want to make a game, consider Godot with GDScript - here's a guide created in Godot to learn GDScript interactively with no programming experience. GDScript is like Python, a very widely used language outside of games, but it is exclusive to Godot so you'll need to transfer it. You can also use C# in Godot, but it's a bigger learning curve, though it is very general and used in a lot of games.

I'm a big Godot fan, but Unity and Unreal Engine are solid. Unreal might have a steeper learning curve, Godot is a free and open-source project with a nice community but it doesn't have the extensive userbase and forum repository of Unity and Unreal, Unity is so widely used there's lots of info out there.

If you did want to go really from scratch, you can try using something like Pygame in Python or Processing in Java, which are entirely code-created (no user interface) but offer lots of helpful functionality for making games purely from code. Very flexible. That said, they'll often run slow, they'll take more time to get started on a project, and you'll very quickly hit a ceiling for how much you can realistically do in them before anything practical.

If you want to go a bit lower, C++ with SDL2, learning OpenGL, and learning about how games are rendered and all that is great - it will be fast, and you'll learn the skills to modify Godot, Unreal, etc. to do anything you'd like, but similar caveats to previous; there's likely a low ceiling for the quality you'll be able to put out and high overhead to get started on a project.

Yeah, I think this is an important thing to be aware of. I 100% get and understand the need to reinforce self-worth outside of what's traditionally pushed. But that's not the whole story, and I don't see much on the other side of it.

It's when you get that anxiety/depression cocktail alongside things, unable to find the motivation to do the things you need to do to feel adequately drained, or unable to do the things that adequately energize.

It's when you fall flat and feel horrible, not because of a corporate agenda, but because real people depended on you and you couldn't show up.

It's when you took the advice, and followed your rhythms of the day, and stopped going against your mental grain.... and then you missed your work deadline, or messed up your work and screwed someone over, or accidentally estranged family members, or didn't get that medical treatment you needed.

It's really important people don't tie up in the self-worth of productivity and corporations - it's really easy to prioritize those because we're told all of our lives that they're worth prioritizing - and that leads to us ignoring our own needs... But unless you're very fortunate, work and productivity are needs, and finding ways to exert energy in a healthy (and often relaxing!) way is important.

I don't know where I'm going with this.

My cynical guess is that's what they're hoping the community will do ("like lemmings, I tell you!" - spez, probably) to drive higher traffic numbers before some announcement or meeting.

Yeah, I really think it's important to not see Lemmy as one singular community, or a lot of important use cases will go ignored.

Not the OP, but in Canada at least, I think you would legally be expected to because common law is (as far as I'm aware) very nearly marriage and is entirely implied by time living together in a conjugal relationship. It might be provincial to determine the actual property laws, though.

I don't have a firm opinion here, but I think the key difference in your case is that a conjugal relationship has some expectation of... Oh I don't know, mutuality? A landlord tenant relationship is a lease agreement. If your roommate didn't sign any kind of lease agreement, they might have a legal case to just not pay you and suffer no consequences (I don't know), but they're not in a conjugal relationship, so there's also no implication of shared ownership.

Without signing lease agreement and being in a conjugal relationship, I think there is a pretty fair case that expecting shared ownership is a fair assumption.

That all said, it's also really up to the individuals to figure that out early, and the deception in the meme suggests that the agency to have that discussion wasn't available, and that's really the part I find problematic here.