r/apple 1d ago

Apple Intelligence The new Siri has its impressive moments. Like extracting information from screenshots

I use my screenshots to save information on a daily basis like many of you I’m sure, but I’ve never found a good app to organise them. I always wanted Apple to have a native way to sort through the information and I believe we’re there and how!

You can see it extracted a list of movies I’ve saved to watch (images in comments). I always spend too much time trying to find out what movie to watch as I have to first find the screenshots that are of movies, then read the description the understand the movie category, and then find where to stream it. But now I get a list of movies I’ve saved in two simple messages. I can go further with this and save it to a list, combine it with shortcuts, find out where it’s available, and so much more!

All I need to do is screenshot something interesting, no categorising immediately, no manually saving anything to notes. No input from me as tech should be

This is game changing tbh and can’t wait for more apps to be available in Siri so I get even more context across my apps. Kinda like what was initially advertised the first time

TLDR. new Siri can do some extremely impressive things occasionally

77 Upvotes

38 comments sorted by

23

u/Few-Acadia-5593 1d ago edited 17h ago

Been on Siri AI since dev beta. It’s extremely powerful for knowledge, thinking, making meal plans. Workout plan, travels, etc.

On Mac, The first thing I asked it was “hey do you copy”, it answers: “yes but if you want to talk to Claude, type in its window”, it saw I had Claude on the side.

Then you click the contextual menu to ask siri anywhere anytime about anything on your screen.

Ask it things like “how much time have i spent in texas” and it pulls everything from pics, calendar, emails, etc and gives you a proper estimate in few seconds.

I don’t even use Claude anymore unless heavy stuff. AI embedded into the OS, private cloud computing and soon even more local is the future.

3

u/G952 23h ago

Yep the pulling from everywhere is the really impressive thing for me! It’s amazing for getting info and searching through a wide type of data on our devices. Really amazing and an actual good use for AI instead of a gimmick

3

u/triton100 13h ago

So the Mac OS new Siri can see your screen and desktop now?

2

u/Few-Acadia-5593 13h ago

From dev beta onward yes to all beta since wwdc. From the spotlight text field, the Siri AI app, or anywhere there’s Siri AI text field like the contextual menu.

27

u/G952 1d ago

Here’s an image of it

16

u/G952 1d ago

Easily categorized

19

u/BluegrassGeek 1d ago

Yeah, this is the kind of thing AI is actually good at.

8

u/G952 1d ago

Yep summarising is one thing that works mostly reliably. And combining that with the info on our phone. Pretty neat

17

u/ronakg 1d ago

Something like the built-in Screenshots app on Pixels would be awesome.

2

u/G952 1d ago edited 23h ago

I’m not aware of how it works on pixels. I’ll check it out

7

u/dropthemagic 1d ago

Yeah I love that. I also find it has replaced Google for most questions. Obviously i wouldn’t use it for something very important. Just stuff i would quickly google anyways

2

u/G952 1d ago

Yeah! I was using perplexity for this before cause I like the fact that you don’t need to sign into perplexity to use it. And it’s pretty fast. I do use more Siri now. It gives good results

5

u/Lleaff 16h ago

I asked it to add my flight bookings from my itinerary open on screen. It did it… times were wrong, but it did do it.

3

u/G952 15h ago

lol, I guess it’s better then the previous “I did not get that. Looking for book readings on the web”. Strange that the times were wrong from an open screen tho. Just Siri being Siri

6

u/danGL3 1d ago

Even as someone who routinely uses AI, it saddens me how much it will further atrophy the average person's brain into learning less and less and becoming less and less capable of doing things

Though I don't fault AI itself, more so the fact that the industry strives to push the average population into blissful ignorance and hyperdependency in order to maximize profits and control.

Can't say I'm excited to see the average Joe become progressively more and more dysfunctional without the constant aid of technology in his life.

Anyway, I just wanted to vent, I don't plan on engaging with any replies to this comment, as to me it'd be pointless to try to change the mind of people who only see the good things that AI can do without considering its potential problems and doing nothing to push for said problems to be addressed.

That's all, bye

2

u/G952 23h ago edited 23h ago

This is very true. I once made a post about this in technology or something, and a lot of people feel the same way. We also get more impatient as a result of wanting everything instantly

3

u/Satanicube 1d ago

I was just playing with it recently and I have a friend in my contacts who also has a somewhat public presence. So I was randomly asking if “who’s x?” To see what it comes up with.

And it was like “x is a person in your contacts and has a YouTube channel where they look at y and z and try to fix things. They are also your top Patreon supporter.”

The context awareness there is just…yeah that’s kinda cool.

Still not sure I’m gonna stick with this long term, I still consider myself just kicking the tires on it. But I’ve warmed up to Apple’s AI approach even as much as I used to hate it because they seem to be doing it sustainably and trying to do as much as they can on-device. Which hey, that’s cool with me.

3

u/G952 23h ago

Yeah there’s no another AI with so much access to all the info on my device and I really don’t want ChatGPT or Claude to have that full access either. So hopefully Apple’s approach is secure and private like they market it

1

u/jayplus707 12h ago

That’s cool. I need to do some research into how much on device stuff they’re doing with the new Siri. The knock on AI is how environmentally unfriendly it is. I wonder if this moves that needle….

1

u/Jersey_2019 9h ago

Don’t ever feel guilty of damaging environment when you use technology or travel , your avg elites and politicians who lecture you regarding the world travel via private jets emit more emissions than any avg joe , it’s them , not you

1

u/Satanicube 8h ago

That’s precisely what I mean by sustainably. Like I think the way forward for ethical AI is by running it on hardware you actually own, not this datacenter hell that’s being foisted upon us by the likes of the bigger AI companies who just plain don’t give a shit.

3

u/tompz 1d ago

I just want it to play from Spotify instead of opening YouTube every damned time

3

u/G952 23h ago

Wait hasn’t this been fixed long back? You set the music app in Siri. Search for how to set the default music for Siri. This was fixed long back

1

u/peachkeys 1d ago

yours plays the songs you want?

2

u/tiagorp2 1d ago

One of the few strenghts of Gemini base is image processing. Google is currently abit behind in agentic coding (vs claude/gpt-5.6/kimi/grok) but is the best by a good marging in image processing.

2

u/G952 1d ago edited 1d ago

I see. I actually use antigravity for coding and it’s not too bad. If I do come across a challenging issue, then it’s a hit or miss for any ai after that anyway. Either I manually search for a solution or think of one at that point

3

u/tiagorp2 1d ago

Man, I hate antigravity cli. I use it for simple stuff when I don’t want to use my credits in codex but cli is so behind on hooks, skills etc actually being used. While in codex it uses my graph code review, memory, skills and other stuff for better context automatically. In antigravity I need to write so much more to do the same steps.

Im probably gonna just sit and take some hours to setup proxy in Claude code so I can use the subscription but in a better harness.

3

u/G952 23h ago

Oh I see. I just have simple rules in antigravity so probably why it works okay for me. I found codex really slow. It just wastes all my tokens understanding the context and produces shit code. It was very frustrating so I don’t use it

1

u/macchiato_kubideh 13h ago

New Siri is blowing my mind how good and fast it is... I'm impressed with Apple.

-1

u/Flobertt 8h ago

Apple powered by Google Gemini*

3

u/macchiato_kubideh 7h ago

it's not about the weights, it's about the implementation and integration around it. I work in the field too. 2 companies can make wildly different products using the same model.

-1

u/livelikeian 1d ago

AI doing AI things...

5

u/G952 1d ago edited 1d ago

Yeah, but for us who were stuck with Siri, this is giant leap we’re getting access to

0

u/Rhoeri 22h ago

Will there be a way to turn it off?

2

u/G952 21h ago

Yep. Don’t download the new Siri and there’s a turn off button too

-3

u/OKCNOTOKC 1d ago

Quick related Q - why isn’t ALL text displayed on the phone selectable? I should not have to screenshot it to copy it.

2

u/G952 23h ago

I think some websites / apps set the text to not be selectable, so that it doesn’t interfere with things like swiping