Selective format-compliant JPEG encryption?

Question

I am working towards building a format-compliant encryption system for pictures. The aim of it is to be able to obscure specific areas of a picture (i.e. faces, car license numbers...) while keeping the rest unaltered. The aim would be, for instance, being able to publish a picture on any given social network, making the "sensitive" regions visible only to a limited number of people, probably a subset of your friends, whom have been granted permission.

I had some (arguable) ideas regarding pixel permutations and BMP, but I have stumbled face-first on the fact that most social networks use exclusively JPEG, and any other format is compressed, essentially ruining the scheme.

So now I am stuck trying to find a new encryption scheme. Ideally, it should have the following properties:

1. Format-compliant: the image should be still a valid JPEG, openable and viewable.
2. Unobtrusive: as much as possible, the encrypted image should not have a chunk of randomly colored pixels on the encrypted areas, but keep somewhat close to what it "should" look like - something like blurring or pixelation, but reversible. The rest of the image should be unaltered.
3. Secure: aside from the crypto perspective, from a perceptual point of view the image should be obscured enough so as to make, e.g. a face unrecognisable without decoding.

2 and 3 are somewhat opposite, so my aim is finding some acceptable middle ground between them.

I have been trying to work with Droogenbroeck and Benedett's "Techniques for a selective encryption of uncompressed and compressed images", which proposes, for every 8x8 square, keeping the DC (aka most significant) coefficient of the DCT, and encoding the rest. However, it works pretty badly for mid-large images: since encryption is done on small squares, and the DC coefficient -which holds most of the energy- is left unaltered, the larger the image is, the "smaller" those 8x8 become in proportion, eventually becoming useless for images over about 500x500. Other than that, most papers I have found use JPEG2000 - which apparently would make everything much easier, but basically isn't supported by anything, so it doesn't seem a reasonable solution for now.

I know this is a bit on the open-ended-questions side, so specifically, my question is: Is there previous work, or any already discussed solution, for this problem or some other similar one? Do you have any ideas about how to approach this?

Thank you for your time, and please let me know if there is I can clarify - ideally I should link pictures of what I mean, but I don't think I'm allowed below a certain rep threshold :(

EDIT

As fgrieu noted, this has an added problem [please see below]. For example, Facebook apparently decompresses and recompresses the image on uploading, which means an additional, unavoidable quality loss. That means "fine-grained" tricks with coefficients – especially the least significant ones – would not work. (Also thanks to Ilmari Karonen for linking the paper!)

The problem pointed by fgrieu is avoidable in other social networks - for instance, Google+ does not recompress the image, so no problems there.

Answer 1

While I haven't heard of previous work on this exact problem, splicing together some standard solutions to a few other problems almost solves this problem.

There is a loophole in the pigeonhole principle: if the output file is bigger than the input image, it is possible to store "extra" information.

Starting from the blurred image you want everyone to see, you somehow want to add more information to it so that the appropriate people can extract an unblurred image.

picture frame

Perhaps you could keep every pixel of that blurred image unmodified, and add something like a picture frame above, below, or all the way around the image to store extra data, like the way dotless QR codes store extra data without modifying the center parts of the image. There's been a lot of work on designing barcodes in such a way that it's possible to recover all the original data, even when the barcode is distorted far worse than simply uncompressing a JPEG and then recompressing with worse quality settings. If you have a very wide bezel picture frame, it may be possible to encode (an encrypted version of) the entire (unblurred) original image file in the frame in the form of a standard barcode. If you create a "residual image", the difference between the (unblurred) original JPEG file vs the blurred JPEG image in the middle of the frame, that residual image should be all-zeros except for the regions that were blurred out, and so (a compressed and then encrypted version of) the residual image should require far fewer bits to encode than the entire original image, and therefore require a narrower frame around the blurred image.

borderless

If you want steganography and you think the frame is too much of a giveaway, there seems to be a bunch of research on digital watermarking designed to be robust enough to survive uncompressing a JPEG, making minor modifications to it, and recompressing. Perhaps you can tweak such watermarking and steganography techniques enough to store (a compressed and then encrypted version of) the residual image. "Hide data in files with easy steganography tools"; "How can you hide information inside a jpg or gif photo?"; etc. If that doesn't give you enough storage space to store the entire residual image, perhaps you could scale up the image (each pixel of the blurred original image expands to 4 pixels of the expanded image), giving more area to encode data. Or perhaps you could scale up the blurred image, and then apply something like DataGlyph halftoning to create an image that from a distance looks like your blurred JPEG file, but the Dataglyph digital halftoning contains (a compressed and then encrypted version of) the residual file or the entire (unblurred) original file.

Answer 2

Selective format-compliant JPEG encryption as you are trying to do it is a great idea, but it won't work... not like this.

To keep the reasons short and simple:

JPEG uses lossy compression (and even lossier recompression). If you really want to create a format-compliant implementation, you'll have to take care that you're independent of any "(re)compression" issues, which might destroy your valuable, encrypted data.

This means you'll have to solve the problem that you can not store as much data in your image as you're encrypting... in simpler words: your algorithm will be lossy by nature too and you will not be able to recover the "encrypted" information back to it's original.

In fact, it's a bit like a reverse pidgeon-hole problem: you will have to see how many pidgeons you can recover after someone closed an unknown number of pidgeon-holes. Whatever you do, it will not be the same number of pidgeons as your original population. Getting back to your image question this means that you will not be able to recover as many pixels as you've encrypted... that's a result of JPEG's lossy compression.

The only alternative option would be to create a new image format which fits your needs. Yet, then the question arises if anyone will adopt your new image format. Looking at what happened to aPNG (animated PNGs), I'm not very optimistic that you'll have a chance - unless some social network(s) push forward and start supporting it. And then you'll still have to see if it really spreads throughout the several programs and tools out there (think: Photoshop support etc.).

Please don't get me wrong: I love the idea you're having there - but as with many good ideas, I'm not seeing it happen for technical reasons. Sorry to say so.

Answer 3

There is this paper, which is some years old in regards of the fast topic development, but it may be a good start.

http://web.cecs.pdx.edu/~fliu/papers/ihmm2015.pdf

The idea here is to somehow encrypt parts or the whole image, while still have some kind of visual clue of the content, a bit like you want to do it. They do it for the whole image, but i guess, thats something which can be worked on in your case.

But the main starting point you already found. Most JPEG Encryptions work on the DCT or DFT, which totally makes sense.

Another maybe interesting approach for you could be to look at "Visual Cryptography". Where images are separated and only viewable in the combination. Its somewhat of a hot topic right now.