Get the top HN stories in your inbox every day.
whatisthiseven
thecupisblue
Now, imagine this future:
You got a friend, spouse or someone close that has hundreds of pictures of you on their phone. Their phone has a "AI chip" that is used to finetune the recognition models and photo models with your AI library. Like Google Photos tags images of people you know, so does the model. It also helps sharpen images - you moved your head in an image and it was a bit blurry, but the model just fixed it, because like the original model had for the moon, it has hundreds of pictures of you to compensate.
One day, that person witnesses a robbery. They try and take a photo of the robber, but the algorithm determines it was you on the photo and fixes it up to apply your face. Congratulations, you are now a robber.
Self-Perfection
Good point.
For the long time digital cameras embedded in EXIF metadata about conditions on which the photo was made. Like camera model, focal length, exposure time etc
Nowadays this metadata should be extended with description of AI postprocessing operations.
jgerrish
> Nowadays this metadata should be extended with description of AI postprocessing operations.
Of course. But to ensure that's valid for multiple purposes we need a secure boot chain, and the infrastructure for it.
To get there we need an AI arms race. People trying to detect AI art with machine learning vs. increasing AI sophistication. Companies trying to discourage AI leaks of company secrets and reduce liability (and reduce the tragic cost of mistakes of course) vs. employees being human.
Or we could have built a responsible and reasonable government that can debate and implement that.
Maybe I'm naive. I'll take responsibility for that.
In the meantime, it's playtime for the AIs. Bring your fucking poo bags, theyre shitting everywhere (1), pack it in, pack it out.
(1) what the world didnt know, was that this was beautiful too.
N19PEDL2
Another option could be to always include the original unprocessed picture in every photo file.
Some image formats (e.g. HEIF) already allow to store multiple images in the same file.
bluesign
Or just simply embed the original data from the sensor
KennyBlanken
The far greater concern is far more mundane.
Photos taken by cell phone cameras increasingly can't be trusted as evidence of the state of something. Let's say you take a picture of a car that just hit a pedestrian and is driving away.
Pre-AI, your picture might be a bit blurry, but say, it's discernible that one of the headlights had a chunk taken out of it; it's only a few pixels, but there's obviously some damage, like a hole from a rock or a pellet gun. Police find a suspect, see the car, note damage to the headlight that looks very close, get a warrant for records from the suspect, find incriminating texts or whatnot, and boom, person goes to jail for killing someone (assuming this isn't the US, where people almost never go to jail for assault, manslaughter, or homicide with a car) because the judge or jury are shown photos from the scene, taken by detectives in the street of the person's driveway, and then from evidence techs nice and close-up.
Post-"AI" bullshit, the AI sees what looks like a car headlight, assumes the few-pixels damage is dust on the sensor/lens or noise, and "fixes" the image, removing it and turning it into a perfect-looking headlight.
Or, how about the inverse? A defense attorney can now argue that a cell phone camera photo can't be relied upon as evidence because of all the manipulation that goes on. That backpack in a photo someone takes as a mugger runs away? Maybe the phone's algorithm thought a glint of light was a logo and extrapolated it into the shape of a popular athletic brand's logo.
hedora
I’d just like them to fix the problem where license plates are completely unreadable by most consumer cameras at night. It’s almost as though they are intentionally bad. (The plate ends up as a blown out white rectangle.)
syrrim
The recent kyle rittenhouse trial had an element that hinged on whether apple's current image upscaling algorithm uses AI, and hence whether what you could see in the picture was at all reliable. The court system is already aware of and capable of dealing with these eventualities.
elicksaur
“Aware of” does not necessarily mean “capable of dealing with”. Forensics is generally bad science, yet gets admitted into court all the time. This occurs despite many legal textbooks, papers, and court opinions highlighting the deficiencies.
0cVlTeIATBs
The question was more general, if the iPad zooming introduced any different pixels (e.g. a purple pixel between red and blue). Or, "uncharged pickles" as the judge put it.
LorenPechtel
It doesn't even need AI to be problematic. Pinch-zoom has no business being used in the courtroom as it inherently can introduce issues. However, a fixed integer ratio blowup of the image shouldn't be problematic. (2:1 is fine. 1.9:1 inherently can't guarantee it doesn't introduce artifacts.)
CiaranMcNulty
Well it's not capable of dealing with it because they found apple's zoom was unreliable and it contributed to the guy getting off
PaulHoule
I thought it was really funny in the 1980s that people in medical imaging were really afraid to introduce image compression like JPEG because the artifacts might affect the interpretation of images but today I see article after article about neural image enhancement and it seems almost no concern that a system like that would be great at hallucinating both normal tissue and tumors.
So far as law and justice goes it is the other way around too. If it is known to be possible that cameras can hallucinate your identity, it won't be possible to use photographic proof to hold people to account.
theptip
It seems fairly easy to bake a chain of custody into your images. Sensor outputs a signed raw image, AI outputs a different signed “touched up” image. We can afford to keep both in this hypothetical future; use whichever one you want.
Once generative AI really takes off we will need some system for unambiguously proving where an image/video came from; the solution is quite obvious in this case and many have sketched it already.
bayindirh
The images generated by SLR and mirrorless cameras are already signed with device embedded keys during EXIF embedding. Every manufacturer sells such verification systems to law enforcement or other institutions to verify such images.
Sometimes there are exploits which extract these keys from the cameras themselves, but I don't hear them nowadays.
One of the older products: https://imaging.nikon.com/lineup/software/img_auth/index.htm
jfim
Until some PM says "why do we have both images, our data shows that 99.5% of users don't use the raw image, let's remove that feature."
7jjjjjjj
Then someone takes a photo of their TV screen. Presto, instant chain of custody for any image you want!
MengerSponge
"I thought what I'd do was, I'd pretend I was one of those deaf-mutes." [1]
If any of you young folks haven't watched Ghost in the Shell, just close this tab and do that.
joncrocks
`Ghost in the Shell: Stand Alone Complex`
Rebelgecko
I feel like with ScarJo lately you either get "big budget movie where she's phoning it in" or "small movie most people won't watch where she's a great actress. Does this fall into either of those categories?
fuckingmorons
[dead]
dzikimarian
Obviously someone who has good enough position to take semi-clear photo and who knows you so well, that has phone full of your face, will not recognize you directly, but will be convinced that you are robber after looking at photo. At this point we can go full HN and assume that you will be convinced anyway, because judge is GPT-based bot.
This "future" is present in current Pixel lineup btw. Photos are tagged as unblured, so for now you can still safely take a selfie with your friends.
CodesInChaos
They should call the chip XeroxAI.
dusted
Basically this.. As "neat" as AI "improvement" is, I don't think it has any actual value, I can't come up with any use-case where I can accept it. "Make pictures look good by just hallucinating stuff" is one of the harder ones to explain, but you did it well..
Another thing, pictures for proof and documentation, maybe not when they're taken but after the fact, for historical reasons, or forensics.. We can't have every picture automatically compromised as soon as it's taken. (Yes, I know that photoshop is a thing, but that's a very deliberate action, which I believe it should be)
jfengel
I think the main use case is "I'm a crummy photographer and all I want is something to remind me that I was there" and "Look at my cat. Look! Look at her!"
That's me. I'm a lousy photographer, as evidenced by all of the photos I shot back when film actually recorded what you pointed it at. My photography has been vastly improved by AI. It hasn't yet reached the point of "No, you idiot, don't take a picture of that. Go left. Left! Ya know what, I'm just gonna make something up," but it should.
I imagine there will remain a use case for people who can actually compose good shots. For the remaining 99% of us, we'll use "Send the camera on vacation and stay home; it's cheaper and produces better pictures" mode.
taffronaut
As a kid I was taking a photo in a tourist spot with a film camera and standard 50mm lens. An elderly local guy grabbed me by the shoulder as I framed the photo. We shared no common language and he (not so gently) pulled me over to where I should stand to get the better shot.
mysterydip
That would actually be a useful feature, I'm aiming the camera but based on what makes "good professional" photos, it suggests "move to the left so you frame the picture well" or "those two people should be more spread out so its not one person with two heads" etc, kindof like lane warnings on cars.
giantrobot
You don't need AI for taking better photos, for most people the phone just automatically taking a burst/video and picking a frame out for the still or stacking frames would be plenty. Lots of photos suck because of shit lighting. A camera intelligently stacking frames would fix a lot of people's photos.
b112
"I'm a crummy photographer and all I want is something to remind me that I was there"
This is fine, and I can take good shots but at the same time? I only care about this level of shot most of the time too!
But then instead of a 20MP image, which:
* takes more space, and ergo, more flash drive space
* more space to store, to backup, to send
* is made 20MP by inserting fake data
Why not have a 2MP image, which is real, and let people's end-use device "fix" it? Because all that post processing can be done when 2x or 4x the view size, too!
Because advertising.
And that's sad. We'd rather think we have a better pic, and destroy the original.
And the space thing is real. Because, that same pic gets stored in gmail with 20 people, backed up, kept in all the devices, and so on!
And the LOL of it all, is that I bet when it is uploaded to facebook... it gets downsized!
edit: in fact, my email app allows me to resize on email, so I downsize that too! Oh, those poor electrons.
Kye
I'm a decent photographer and still use my phone for this. It's good enough, and can even skip the AI stuff if I want to. Or even better: I can keep the AI stuff in the raw and edit its impact on the final photo later.
rjzzleep
Interestingly enough one of the reason Sonys flagships perform really badly in comparisons is because they are weak at computational photography. So even when the sensor is great it looks too real, which people don't like.
avereveard
> it looks too real
Yeah I've a phone with a great camera, nature shots are great, but people don't like themselves in these photos. When pressed they talk about the defects on skin and theets and eye position... Their phone beauty filters created in their mind a fake mental image of themselves and they dissociate from their real images.
It's weird. My mom brand fidelity is because Huawei specific algorithm is part of her self.
shawnz
How about using AI for sensor fusion when you have images from multiple different kinds of lenses (like most smartphones today)? I was under the impression this was the main reason why AI techniques became popular in smartphone cameras to begin with
gabrielhidasy
I'm not aware of much fusion happening between different lenses (although I saw an article using that for better portrait mode bokeh), but AI is used to stack multiple images from the same sensor. You can do de-noise, HDR and other stacking stuff with clever code, but AI just makes it better.
montagg
Good for situations where you aren’t expecting or care about realism in this detail. AI hallucinations will be amazing for entertainment, especially games.
worrycue
I want game content generation by AI, like for dungeon generation in an ARPG - it likely won’t be as good as hand crafted level by a developer but it should be more interesting than the current techniques where prefab pieces of a dungeon are randomly put together.
adrr
Removing noise from low lights pictures, or removing motion blur from shaky hands. Lots use cases for “ai” or computational photography.
GuB-42
> We can't have every picture automatically compromised as soon as it's taken.
Isn't it a good thing for privacy?
dusted
I think it's neutral.. Just as incriminating as true photos can be (at least there might be some moral highground if you're into that sorta stuff), for AI faked pictures.. You may have no choice but be incriminated by photos that lie..
KyeRussell
I genuinely can’t recall people saying “hallucinate” with any regularity - in the context of “AI” - until people started talking about ChatGPT.
So, we’ll see what people say in a year.
naniwaduni
The term has been around in this context since at least 2018[1], and indeed I have chat logs from 2019 talking about how mtl [machine translation] hallucinates, so no, this has been what people have been calling it for a while now. Perhaps what you're seeing is just rising awareness that this is a weakness of current-gen ML models, which is great, now even monoglots get to feel my pain :V
[1]: https://www.wired.com/story/ai-has-a-hallucination-problem-t...
kuboble
I think it started a bit earlier - already with image generation AI like dall-e.
wakeupcall
I don't already fully trust the images, audio and videos I take with the phone.
I'm working close to HW and I actively use the camera/picture and videos for future reference and debugging. It's small, fits in your pocket, and the bloody thing can record at 240fps to booth!
Until you realize there's so much post-processing done on the images, video and audio you can't really trust and can't really know if you can turn it all off. The reality is that if you could, you'd realize there's no free lunch. It's a small sensor, and while we had huge improvements in sensor and small lenses, it's still a small sensor.
Did the smoothing/compression remove details? Did the multi-shot remove or add motion artifacts you wanted to see? Has noise-cancelling removed or altered frequencies? Is the high-frame rate real, interpolated, or anything inbetween depending on light just to make it look nice?
In the end, they're consumer devices. "Does it look good -> yes" is what thrums everything in this market. Expect the worst.
giantrobot
> Did the smoothing/compression remove details? Did the multi-shot remove or add motion artifacts you wanted to see? Has noise-cancelling removed or altered frequencies? Is the high-frame rate real, interpolated, or anything inbetween depending on light just to make it look nice?
This has been true of consumer digital cameras for 25 years. It's not new to or exclusive to smartphone cameras. It's not even exclusive to consumer cameras as professional ones costing many times more also do a bunch of image processing before anything is committed to disk.
kuschku
With even an a6000 (you can get it used for about 200 bucks) you can get high quality RAW images without any postprocessing.
And they actually look good!
No phone can deliver that, even today.
charrondev
I don’t know about android, but at least with my iPhone I’m pretty sure there are apps that can capture raw sensor data. Additionally I do have the ability capture Apple ProRAW format at of the photos. I don’t actually know if these images are still processed though.
nuancebydefault
Raw format? That means without any debayering applied? That would mean every pixel has only either r, g or b information and not combined. Be aware that there exist different debayering algorithms, sometimes applied depending on the context. Also, without any correction for sensor calibration? That would mean every sensor has a different raw image for the same input. My point being, without application of algorithms, the info captured by the sensor does not really look like an image.
wefarrell
I can imagine there will be a lot more “ghost” pictures like this one: https://www.reddit.com/r/Ghosts/comments/11jvwy4/after_years...
as AI tries to infer images of people where they aren’t really present.
sowbug
I don't know if you even need AI for this.
"You just took a picture of the Eiffel Tower. We searched our database and found 2.4 million public pictures taken from the same location and time of day. Here are 30,000 photos that are identical to yours, except better. Would you like to delete yours and use one of them instead?"
dylan604
royalty free I'd assume you meant as well, or are you pitching a new SaaS model for stock photos?
astrange
There are technically not royalty free images of the Eiffel Tower at night because the lights' owners consider them copyrighted.
A4ET8a8uTh0
This is probably one of the few good things about current meltdown and no low rates. No more silly Peletoneque startups.
edit: for a while anyway
sowbug
Haha, the idea's all yours. Let me know how it goes.
hotpotamus
There was a pretty neat Google project a few years back that showed time-lapse videos of buildings under construction created entirely through publicly posted images that people had happened to take at the same spot over time.
MonkeyMalarky
I wonder if that'll ever cause legal problems in the future. Sorry, that photo someone took where the accused was in background at a party some years ago? He was kinda blurry and those facial features have been enhanced with AI, that evidence will have to be thrown out. Or maybe the photo is of you, and you need it as an alibi..
Grambo
This is actually exactly what happened during the Kyle Rittenhouse case. A lawyer for the defense tried to question video evidence because of AI being used to enhance zoomed shots.
dtx1
No that was what the mainstream media lied to you about what happened in the rittenhouse case. One of several instances where one could see fake news and straight up lies be spread in real time.
What actually happened was that a police officer testified that using pinch to zoom on his iPhone he saw kyle point his rifle at the first person assaulting him. Mind you we were talking about a cluster of around 5px. The state wanted to use an iPad to show the jury using pinch to zoom that same "evidence" because using proper zooming without an unknown interpolator algorithm the defense using an expert witness showed that this was not the case. No one in that courtroom understood the difference between linear and bicubic interpolation.
The defense did not understand it either so they tried to explain to the judge that the iPad Might use AI to interpolate pixels that aren't there and that the jury should only use the properly scaled video the court provided not an ad hoc pinch to zoom version in an iPad with unknown interpolation.
Thankfully the judge told the state to fuck off with their iPad but the mainstream media used the bad explanation of the defense against kyle when the reality was that the state basically tried to fake evidence live on stream using an iPad to zoom in.
BTW I'm German so I don't have a horse in the political race but I watched the Trial on live stream and saw the fake news come out while watching
formerly_proven
"Good sensors are expensive"-fun-fact: Mid-range CCTV cameras often have bigger sensors (1/1.8" or 1/1.2") and much faster lenses than an iPhone 13 Pro Max (1/1.9" for the main camera). The CCTV camera package is of course far bigger though. But still kinda funny in a way.
Edit: And the lenses on these are not your granddads computar 3-8/1.0, either. Most of the CCTV footage we see just comes from old, sometimes even analog, and lowest-bidder installations.
undefined
gostsamo
Bruce Sterling I think had a story in that direction. A polaroid camera producer would develop photos which would've been algorithmically enhanced so that their clients consider themselves better photographers and their cameras superior. I'm regularly updated for it for the last few years when cameras are more and more their software.
Edit: fixed the author's name. Cannot find the exact story though.
stubish
This has in some ways been happening for decades. There are a few countries where the way to take a good portrait of a person is to over expose the photo, so skin tones are lighter. People bought the cameras and phones that did this by default (by accident or design in the 'portrait mode' settings). They didn't want realism.
Diggsey
I think they're probably right about the AI-sharpening using specific knowledge about the moon... However, they are wrong about the detail being gone in the gaussian-blurred image.
If they applied a perfect digital gaussian-blur, then that is reversible (except at the edges of the image, which are black in this case anyway). You still lose some detail due to rounding errors, but not nearly as much as you might expect.
A gaussian blur (and several other kinds of blur) are a convolution of the image with a specific blur function. A convolution is equivalent to simply multiplying pointwise the two functions in frequency space. As long as you know the blur function exactly, you can divide the final image by the gaussian function in frequency space and get the original image back (modulo rounding errors).
It is not totally inconceivable that the AI model could have learned to do this deconvolution with the Gaussian blur function, in order to recover more detail from the image.
Terretta
Author tested for this by doing the experiment again with detail clipped into highlights, completely gone, model detail was added back.
> To further drive home my point, I blurred the moon even further and clipped the highlights, which means the area which is above 216 in brightness gets clipped to pure white - there's no detail there, just a white blob - https://imgur.com/9XMgt06
> I zoomed in on the monitor showing that image and, guess what, again you see slapped on detail, even in the parts I explicitly clipped (made completely 100% white): https://imgur.com/9kichAp
Groxx
While I think this is a great test, I'm not really sure what that second picture is supposed to be showing. Kinda seems like they used the wrong picture entirely.
Izkata
Second image is a video, shows them zooming in and how it switches from the blob to detail
mgraczyk
I watched the video and in this case the "recovered" detail is clearly natural to me. The original case does look like some kind of moon-specific processing, but this one with clipped highlights seems natural and can be achieved using classical CV.
jitl
What? Clipped means gone - the pixel is FFFFFF - how can CV look at a FFFFFF pixel, surrounded by FFFFFF pixels, and get out a moon-looking pixel?
xmcqdpt2
> As long as you know the blur function exactly, you can divide the final image by the gaussian function in frequency space and get the original image back (modulo rounding errors).
Those rounding errors are very important though. The Gaussian function goes to zero very quickly and dividing by small numbers is not a good idea.
If your deconvolving a noise free version of the original that also doesn't have any saturated pixels (in the black or white direction) then you can get the pretty close to the original back. I don't think this applies here because the OP is taking a picture of a screen that shows the blurred version, so we've got all kind of error sources. I think the OP is right: the camera is subbing in a known picture of the moon.
It would be interesting to see what happens with anisotropic blur for example, or with a picture of the moon with added fake details (words maybe?) and then blurred.
undefined
raincole
> However, they are wrong about the detail being gone in the gaussian-blurred image.
Well yes, but he also downsampled the image to 170x170. As far as I know, downsampled information is strictly lost, and unrecoverable without an external information source (like an AI model trained with pictures of moon).
drhagen
I'm too lazy to downscale it myself, so here's a 180x180 picture of the moon from WP [1]. This looks about the same as the Samsung result [2]. They are not getting the original detail, but they are getting the detail they should expect if Samsung simply deconvolved the blurred image.
[1] https://upload.wikimedia.org/wikipedia/commons/thumb/2/2b/Lu...
geor9e
>If they applied a perfect digital gaussian-blur, then that is reversible
Not true. Deconvolution is a statistical estimate. Think about it. When you blur, colors get combined with their neighbors. Statistically this moves toward a middle grey. You're compressing the gamut of colors towards the middle, and thus losing information. Look at an extreme case - 2 pixels of mid-grey. It can be deconvoluted to itself, to a light and dark grey, or to one black and one white. All those deconvolutions are equally valid. There's no 1-to-1 inverse to a convolution. If you do a gaussian blur on a real photo and then a deconvolution algorithm you'll get a different image, with an arbitrary tuning, but probably biased towards max contrast in details and light noise, since that what people expect from such tools and what most real photos have. But, just like A.I. enhanced images, it's using statistics when filling in the missing data.
vishal0123
geor9e
Wow, that is so cool, and such a good writeup. I like the analogy to an encrypted file, and the key being the exact convolution. The amount of information lost is the amount of information in the key.
I wonder if there is some algorithmic way to find the key and tell if it's correct - some dictionary attack, or some loss function that knows if it's close. Perhaps such a thing only works on images that are similar to a training set. It wouldn't work on black and white random dots, since there'd be no way for a computer to grade or know statistics for which deconvolution looks right.
Cyykratahk
But the AI should not have learned to apply a Gaussian deconvolution kernel. If anything it should be applying a lens-based bokeh kernel instead. A true lens blur does not behave like a Gaussian blur.
top_sigrid
While the information might be recoverable, the information is not seen by the camera sensor. Hence I think the argument in the post stands. Some AI model/overlay magic is happening, pretending to display information the sensor simply did not receive.
squeaky-clean
You're forgetting it was also downscaled to 170x170, and later had the highlights clipped. Both are irreversible.
ipv6ipv4
This is incorrect. The frequency domain inverse of the Gaussian ends up yielding a division by zero. There is no inverse for the Gaussian.
colanderman
That is mathematically true but not practically. Though indeed the Gaussian kernel has lots of zeros [1], in actuality, (a) the zeros themselves are at points, not regions, and therefore of little consequence, and (b) in practice the noise generated from reamplifying frequencies near these zeros can be minimized via techniques such as Wiener deconvolution [2].
[1] https://en.wikipedia.org/wiki/Window_function#Gaussian_windo...
godelski
They didn't claim invertible. The de-gaussinization is a reversible process albeit not invertible. I actually say more in this comment
ibreakphotos
Hey all, it's the author of the reddit post here. First of all, let me say that I don't usually frequent HN, but the comments on here are of such high quality, that I might need to change that. I got semi-depressed on reddit, with people misattributing statements and, in general, not being overly, uh, skeptical :)
That being said, there were a few comments on here about gaussian blur and deconvolution, which I would like to tackle. First, I need to mention that I do not have an maths/engineering background. I am familiar with some concepts, as I've used deconvolution via FFT several years ago during my PhD, but while I am aware of the process, I don't know all the details. I certainly didn't know that the image that was gaussian blurred could be sharpened perfectly - I will have to look into that. In fact, I used gaussian blur to redact some private information (like in screenshots), and it's very helpful to know if I haven't redacted anything and the data is recoverable. Wow.
I would love to learn more about the types of blur that cannot be deconvoluted.
However, please have in mind that in my experiment:
1) I also downsampled the image to 170x170, which, as far as I know, is an information-destructive process
2) The camera doesn't have the access to my original gaussian blurred image, but that image + whatever blur and distortion was introduced when I was taking the photo from far away, (whatever algo they are using doesn't have access to the original blurred image to run a perfect deconvolution on)
3) Lastly, I also clipped the highlights in the last example, which is also destructive (non-reversible), and the AI hallucinated details there as well
So I am comfortable saying that it's not deconvolution which "unblurs" the image and sharpens the details, but what I said - an AI model trained on moon images that uses image matching and a neural network to fill in the data.
Thank you again for your engagement and your thoughtful comments, I really appreciate them, and have learned a lot just by reading them!
lucb1e
> In fact, I used gaussian blur to redact some private information
Absolutely never do that. I honestly don't understand why people still do, given that it's obvious that low levels of blur can be reversed why even risk guessing until what point someone might be able to recover anything? Just censor it, draw over it with an opaque tool, and save it in a format that won't store layers or undo history or something (the riskiest format being pdf).
If you don't like how that looks, the alternative is to replace the information and then blur it. They can unblur but will find an easter egg at best.
Personally, I censor instead of blurring a replacement, but I balance between low contrast and not hiding the fact that information was removed. A stark contrast distracts and looks ugly. E.g., for black text on a white background, I'd pick a light/medium gray (around the average black level of the original text, basically).
rhtgrg
> save it in a format that won't store layers or undo history or something
For eliminating such risk, just screenshot your censored content and use that image.
tethys
Should be rather easy to prove if Samsung is really able to „unblur“ an image in that way: use something else than an image of the moon as starting point and apply the same steps, i.e. down sizing and blur, then take a photo and see if it’s able to recover details.
SamBam
I just wanted to say that the experiment at the end where you had half the moon and the whole moon was brilliant, and perfectly illustrated the problem in a single picture. If anyone hasn't seen that they should.
quitit
It would be interesting to see how this mode handles foreground objects such as an aeroplane or clouds.
Does it just overdraw it (i.e. erase it), apply texture over the foreground element, or fail altogether?
This scenario is probably why other vendors don't go so far as to fake such images with texture overlays.
furyofantares
I took a max-(non-optical)-zoom photo of a rabbit in my yard a while back using an iphone, then further enlarged the result to see how it did - in the details it looked like an impressionistic painting of a rabbit, facing the camera and looking left. The actual rabbit was looking away from the camera and to the right. The eye and face were not visible.
https://i.ibb.co/Kz7Sbm2/8-EA85-C12-5-B11-44-D8-9566-461-C98...
kuschku
Look at pictures on social media in recent months, pretty much every single image looks like this. If you zoom in, they're all inaccurate impressionist paintings. They only look good at original size.
It's absolutely painful and encourages me yet again to use my mirrorless camera even more.
sushisource
At this point I wish we had legislation requiring a "turn off AI bullshit" option for any camera. Every time I've taken a picture of myself using the camera app built-in to WhatsApp, it blurs the shit out of my face in an attempt to hide blemishes or whatever, and (in a weird sort of reverse-vanity) it really annoys me that it gives the impression I care about looking un-blemished, which I do not. AFAICT, there is no way to turn this off.
chroma
Is there any camera that doesn't let you disable advanced image processing algorithms? It'd have to be some sort of locked-down phone, because there are many camera apps for both Android and iPhone. Also the default camera apps do allow you to disable these features. Nobody is forcing anyone to use fancy image processing algorithms. Having the government intervene for something like this seems like overkill.
astrange
Most any image filtering will soften faces even if that's not the point (either by accidentally removing noise or because image stacking doesn't work if there's noise.) If you want it to look "natural" you actually have to make an effort to add the noise back later.
rwalle
While I understand what the author tries to say, I have to point out that ship has long sailed. Samsung just pushed it a bit too far and slapped a "scene optimizer" label on it.
AI has been used in "cell phone photography" for a few years, at lease since Pixel 2 where a mediocre sensor produced much better pictures than what people expect (maybe there are other players who did this even earlier). And every manufacturer started doing it, including Apple. Otherwise, do you think "night mode" is just pure magic? Of course not, algorithms are used everywhere.
How do you define "fake"? In podcasts, Verge editor Nilay Patel has asked various people "what is a photo", because the concept of a "photo" has become increasingly blurry. That is the question the author is asking, and people may have different answers from the author's.
squeaky-clean
> Otherwise, do you think "night mode" is just pure magic?
Night mode definitely uses some AI but most of the result is from stacking frames. Samsung here did not label it as a "scene optimizer". Their marketing just calls it Space Zoom. The only disclaimer they provide is "Space Zoom includes digital zoom, which may cause some image deterioration."
tooltalk
According to Samsung -- and I just confirmed it on my S22 under Camera -> Camera Setting -- it's called "Scene Optimizer"[1]:
[ Overview of moon photography]
Since the Galaxy S10, AI technology has been applied to the camera so that users can take the best photos regardless of time and place.
To this end, we have developed the Scene Optimizer function, which helps AI to recognize the subject to be photographed and derive the optimal result.
From Galaxy S21, even when you take a picture of the moon, ai recognizes the target as the moon through learned data, and multi-frame synthesis and deep learning-based ai technology when shooting. The detail improvement engine function that makes the picture clearer has been applied.
Users who want photos as they are without AI technology can disable the optimum shooting function for each scene.
kramerger
Note that this behaviour is limited to scene mode, which has a moon shoot mode. You can always use the normal or pro mode where the pictures are not magically enhanced.
Is is ridiculous that OP consider this "cheating". Most people just want a nice picture and don't give a damn about AI.
eternalban
Photo is an interesting word. It's meaning is clarified by other words, such as photorealism, photofinish. These words will (strictly) lose their meaning if photograph simply means image captured and processed by a device.
Curiously and revealingly, the political word photo-op stands alone in this photo- parade of words in the age of photo-imaginings. The universe does indeed have a sense of humor.
bsder
Using algorithms to take multiple pictures and stack them together is fine. The information is real, exists, and objective. People in the background won't (for example) suddenly be facing the other way because of the algorithm.
The problem is that AI isn't just interpolating data. It is wholesale adding extra data that simply doesn't exist. The person in the background is facing left, but the sensor couldn't possibly have captured that detail even after multiple images--it was a coin flip that the AI made.
The issue is that, like privacy, most people won't care ... until they do. By that time, it will be too late.
nomel
> People in the background won't (for example) suddenly be facing the other way because of the algorithm.
Someone here included an example where it does do something like this: https://news.ycombinator.com/item?id=35109568
dm319
The software technology in the original pixel cameras were using multiple frames of varying exposure to allow for impressive dynamic range in images while still retaining colour and contrast. This is quite a difficult thing to do as requires precise understanding of what the 'edge' of an object is, and I think that is what AI was used for. This stacking technique is also used for night exposures.
I'm sure that they have started using AI to fill in details more recently, but this is just to point out clever use of multiple exposures and AI can help without faking detail.
giraffe_lady
eg "Do We See Through a Microscope?" https://philpapers.org/archive/HACDWS.pdf
elwell
> the concept of a "photo" has become increasingly blurry
nice.
segfaultbuserr
This problem is not new. In 2019, Huawei introduced a special image processing feature in its smartphone camera app, the "Moon Mode" (opt-in). Missing details are added to the moon photos via machine learning inference from a pre-trained model. Huawei then started marketing these processed images as a showcase of its new smartphone's photography performance. In China, it was widely criticized by tech reviewers [1][2] as misleading, and "Moon Mode" became a running gag among tech enthusiasts for a while.
It seems that Samsung simply adopted the same tactic to compete...
On the Huawei "Moon Mode" controversy, one can even find a research paper [3] published in a peer-reviewed social studies (!) journal, Media, Culture & Society:
> This is where the controversy began: Chinese tech critic Wang’s (2019) posting on Weibo, the Chinese equivalent of Twitter, made quite a splash. In his post, Wang put forward a shocking argument: he said that Huawei’s Moon Mode actually photoshops moon images. He contended that, based on his self-conducted experiments, the system ‘paints in pre-existing imagery’ onto photographed takes, re-constructing details that are not captured in the original shots. Huawei immediately refuted these claims, stressing that the Moon Mode system ‘operates on the same principle as other Master AI modes that recognize and optimize details within an image to help individuals take better photo'
[1] https://www.androidauthority.com/huawei-p30-pro-moon-mode-co...
[2] https://www.phonearena.com/news/Is-the-Moon-Mode-on-the-Huaw...
[3] https://journals.sagepub.com/doi/full/10.1177/01634437211064...
quitit
This couldn't be the same Huawei who has been caught repeatedly using DSLR stock photography in their marketing materials, while claiming the images were taken by their smartphones.
https://petapixel.com/2020/04/21/huawei-accidentally-claims-...
troyvit
It would be so fun to hook a phone up to a telescope and take a picture of, say, Jupiter, and see if it overlays the ringed planet with the moon's characteristics.
geor9e
My hypothesis is that the neural network was trained on a lot of labeled photos, so somewhere inside the network, when you see the moon, it has some moon=0.95 confidence number, and whatever label has the highest confidence, it tries to bring it up to 1.0 akin to how deepdream makes images of spaghetti have more dog faces. Samsungs marketing department interprets that as technically enhancement of images and not faking the moon specifically. So perhaps if it sees Jupiter, it will try to make it more jupitery.
dcdc123
I knew I saw this before.
hettygreen
This is just make hipsters get into old point and shoot digital cameras... good thing I kept my Canon A540.
All the subtle trickery manipulation that the smart phone's doing to reality is concerning. Smoothing people's faces, making their eyes pop, enhancing the shit out of the colours, and now plopping fake objects overtop of the real ones.
Future concerns of this technology should range from a low-key disconnect from reality, to the complete inability to photograph certain objects or locations.
Imagine dusting off a 30 year old digital camera, finding some AA batteries to put in it, snap a selfie and then realizing just how ugly we all are and how washed out the polluted world actually looks without a bunch of narcissism-pandering enhancements.
sorenjan
Not sure about hipsters, but it's apparently somewhat of a trend with young people.
"The Hottest Gen Z Gadget Is a 20-Year-Old Digital Camera
Young people are opting for point-and-shoots and blurry photos."
https://www.nytimes.com/2023/01/07/technology/digital-camera...
javajosh
They are realizing that these things are all toys and fashion so you might as well save money and just buy old stuff and above all take photos.
I think it's great. It's the exact opposite of the person who spends all their time gear shopping and never using the gear.
hef19898
I can highly recommend older Nikon DSLRs and older AF-D or manual focus AI-s lenses. Youbhet full frame bodies, the awesome D700 for example, and a very decent set of lenses rivaling the holly trinary for around the same price you pay for a Z50 with a kit lens. Nothing wrong with a Z50, it is a great camera. But the old, used gear is just such better value IMHO. Ai-s lenses are getting more expensive so since a couple of years so.
The remaining budget can go into going to nice places to shoot nice photos, and to print them nicely and large enough to put on your wall choice.
I picked Nikon, but I assume Canon works as well.
tiborsaas
I'm 40 and did the same thing, bought a cheap analog camera with black and white film to shoot fun photos on my birthday party :) The film is still at the lab, but I guess it's worth the wait.
vel0city
I love my Canon AE-1 but dang it's expensive to actually use.
The G9x Mark II, Sony RX100, Panasonic LUMIX and similar 1" sensor cameras are awesome though and I don't think they've gotten too crazy with computational photography. I imagine some color processing modes might be doing a bit of work though.
seydor
> Smoothing people's faces, making their eyes pop, enhancing the shit out of the colours,
Our brains do far worse stuff with our memories. Not sure how relevant 'high fidelity' is to people who mostly use phones for memories
worrycue
That’s what photos used to be good for. They don’t fudge stuff like our brains.
paulcole
What about and white photos though?
Retric
I regularly take photos of text etc because I am not going to remember it. If a photo of a config password is AI fucked into showing the wrong digits, there’s a real problem.
germinalphrase
“Future concerns of this technology should range from a low-key disconnect from reality…”
This phrase could cast a broadening net with each year’s new tech.
undefined
drhagen
> I downsized it to 170x170 pixels and applied a gaussian blur, so that all the detail is GONE. This means it's not recoverable, the information is just not there, it's digitally blurred
Strictly speaking, applying a Gaussian blur does not destroy the information. You can undo a Gaussian blur with a simple deconvolution, which is something I would expect even a non-AI image enhancement algorithm to do (given that, you know, lenses are involved here).
I'd like to see what detail can be "recovered" with just the downsizing, which DOES destroy information.
xmcqdpt2
Well the op did downsize so details had to be reconstructed. Also the noise from having the image being projected through a screen and then a retaken through the camera sensor means that it isn't just your standard perfect convolution.
padjo
They say they also clipped all whites above a certain level too. That’s just information that’s been destroyed and then invented by the AI right?
notatoad
if you downsize an image to 170x170px and then blow it up so it's visible to a camera from across the room without any sort of blurring, it's not going to look like anything and the camera's object detection won't recognize it as the moon - it's just going to look like a huge pixel grid.
Jabbles
I am not quite so confident. I would like to see an experiment to test how badly you can distort an image of the moon before the AI stops recognising it.
themoonisachees
I'm not so confident either, especially when you consider than whatever input the NN gets is probably downscaled to hell from however many gigapixels the sensor has, otherwise good luck running it on a phone with 8GB of ram.
undefined
ibreakphotos
OP here again.
I photoshopped one moon next to another (to see if one moon would get the AI treatment, while another would not), and managed to coax the AI to do exactly that.
This is the image that I used, which contains 2 blurred moons: https://imgur.com/kMv1XAx
I replicated my original setup, shot the monitor from across the room, and got this: https://imgur.com/RSHAz1l
As you can see, one moon got the "AI enhancement", while the other one shows what was actually visible to the sensor - a blurry mess
I think this settles it.
simoneau
Famously, upscaled Obama: https://twitter.com/Chicken3gg/status/1274314622447820801
I picture someone 20 years from now trying to find out what their parent really looked like when they were young. The obviously smoothed-out face filters are already giving way to AI-powered homogenization. And the filtering is moving deeper down the stack from the app to the camera itself. There will be no "original".
CharlesW
This issue has been known for a few years. Here's a more thorough analysis from January 2021:
"Is the Galaxy S21 Ultra using AI to fake detailed Moon photos?": https://www.inverse.com/input/reviews/is-samsung-galaxy-s21-...
Note that the author decides that Samsung's photos are "not fake", in the sense that they were not doctored after being taken with the phone. However, the article decisively proves that they're being heavily doctored in-camera.
Another test would be to shoot RAW + JPEG if the camera supports it. A true RAW image would reveal what the sensor is actually capturing.
can16358p
Depends. For example Apple can save ProRaw format which DOES include the postprocess magickery, but uncompressed and having more detail for post.
BUT, it also provides apps with true RAW, which is true (or very close to true) raw.
Two different definitions of "raw" is apparently possible even on the same device.
CharlesW
> For example Apple can save ProRaw format which DOES include the postprocess magickery, but uncompressed and having more detail for post. BUT, it also provides apps with true RAW, which is true (or very close to true) raw.
Thank you for that detail. IMO, that makes "ProRAW" a really unfortunate naming decision on Apple's part. And I think you're saying that if you shoot in RAW with, say, Halide, the result will be an actual RAW (pre-demosaic) file in DNG format.
A couple of interesting articles on the topic: (1) https://lux.camera/understanding-proraw/ (2) https://www.austinmann.com/trek/iphone-proraw
neogodless
Pentax K3-II 300mm: https://cdn.discordapp.com/attachments/1010562706237038633/1...
Sensor: 23.5 x 15.6mm 24MP
S21 Ultra: https://cdn.discordapp.com/attachments/1010562706237038633/1...
Telephoto sensor: 3.3 x 4.3mm 10MP (240mm equivalent)
Compare side by side: https://imgur.com/a/QwnV99D
saiya-jin
I have a similar photo done on S22 ultra on March 6 this year, and neither look like your (position of lower right mega crater but also the rest). So its not simple 'photoshop-into-predefined-nice-image'.
I can clearly see that most folks here don't actually own discussed devices (which is fine, its US-based HN, a bastion of iphone and many Apple employees dwell here and uncritical appreciation of Apple is very evident in every single related thread). I've used its 10x zoom extensively over more than a year, it simply blows all other phones away easily for that kind of situation (more than those rather weak 3x zooms available everywhere). Family photos, wild animals, nature, anything you want to come closer, otherwise the scene is tiny dots in the center like on other phones. It works really well for what it is, with obvious unavoidable physical limits.
Overall this phone made me put my fullframe Nikon D750 away on a day I bought it. I took it 'just to be sure' on vacation to Egypt last year, didn't touch it a single time. Most often it doesn't produce strictly as good images but a) they are good enough to be viewed on phones side by side easily, basically as good as fullframe there and sometimes even much better, ie handheld photos in the night of dark scenes, fullframe is utterly lost without tripod, and b) it weights 0 and takes 0 extra space (and cost 0 instead of many thousands for modern camera with big sensor), since I have phone with me always anyway.
Tried exactly the steps as author of article, couldn't reproduce it a bit, tried various mega zooms, his various original photos, dark room etc. Blur remained blur, nothing added. I mean at this point everybody acknowledges any decent phone is painting quite a bit (ie iphone taking other side of bunny than reality, thats a fine example) and I am sure Samsung is doing their part as they have the literal android flagships.
mannykannot
To be clear, are you saying the difference in orientation was introduced by the camera, or is there something more subtle going on here?
neogodless
Orientation is because photos were taken months apart.
This is to get an idea of quality, with the APS-C camera having a much larger (26x area) sensor and better optics to work with.
My impression is... S21 Ultra "space zoom" is, at best, a good party trick. But if you zoom in, the quality is still nearly garbage. Not an objectively "great photo of the moon."
saiya-jin
You realize that these photos are to be compared with other phones and not some apsc on tripod, thats a ridiculous premise for everybody understanding 101 of photography that not even Samsung during any release was claiming to beat.
It still shows you much more details than visible via eyes, so yeah its a party trick (what else would moon shot on phone be), but pretty darn great at that (I haven't seen so many people with :-O since iphone 1 release when showing this... then they quickly try their top iphones and xiaomis and end up consistently with a small white blob).
There is one aspect that this phone wins at easily - it can take that moonshot (TBH it can be a bit sharper than yours) while handheld, pretty consistently. Good luck trying that with your apsc with such a long shutter, it will consistently end up in just a blur. Software often beats raw hardware even these days.
Get the top HN stories in your inbox every day.
Imagine this future:
Sensor quality in phones goes down, AI makes up for it because good sensors are expensive, but compute time in the cloud on Samsung owned servers is cheap. You take a picture on a crappy camera, and Samsung uses AI to "fix" everything. It knows what stop signs, roadways, busses, cars, stop lights, and more should look like, and so it just uses AI to replace all the textures.
Samsung sells what's on the image to advertisers and more with the hallucinated data. People can't tell the difference and don't know. They "just want a good looking picture". People further use AI to alter images for virtual likes on Tiktok and Insta.
This faked data, submitted by users as "real pics in real places" is further used to train AI models that all seem to think objects further away have greater detail, clarity, and cleanliness than they should.
You look at a picture of a park you took, years before, and could have sworn the flowers were more pink, and not as red. You are assured, by your friend who knows it all, that people's memories are fallible; hallucinating details, colors, objects, sizes, and more. The image, your friend assures you further? "Advanced tech captured its pure form perfectly".
And thus, everyone will demand more clarity, precision, details, and color where their eyes don't remember seeing.