PHP Encrypt Streamed File From Javascript

I am developing a File Uploader for big file. Upload from HTML script and send by byte from Javascript using ArrayBuffer and Unit8Array to PHP. The PHP script will stream the file and save it into folder.

Here's my Javascript looks like

function upload(fileInputId, fileIndex)
    {
        var file = document.getElementById(fileInputId).files[fileIndex];
        var blob;
        var reader = new FileReader();
        reader.readAsBinaryString(file); 
        reader.onloadend  = function(evt)
        {
            xhr = new XMLHttpRequest();

            xhr.open("POST", 'upload.php?name=' + file.name, true);

            XMLHttpRequest.prototype.mySendAsBinary = function(text){
                var data = new ArrayBuffer(text.length);
                var ui8a = new Uint8Array(data, 0);
                for (var i = 0; i < text.length; i++){ 
                    ui8a[i] = (text.charCodeAt(i) & 0xff);

                }

                if(typeof window.Blob == "function")
                {
                     blob = new Blob([data]);
                }else{
                     var bb = new (window.MozBlobBuilder || window.WebKitBlobBuilder || window.BlobBuilder)();
                     bb.append(data);
                     blob = bb.getBlob();
                }

                this.send(blob);
            }

            var eventSource = xhr.upload || xhr;
            eventSource.addEventListener("progress", function(e) {
                var position = e.position || e.loaded;
                var total = e.totalSize || e.total;
                var percentage = Math.round((position/total)*100);
            });

            xhr.onreadystatechange = function()
            {
                if(xhr.readyState == 4)
                {
                    if(xhr.status == 200)
                    {
                        console.log("Done");
                    }else{
                        console.log("Fail");
                    }
                }


            };

            xhr.mySendAsBinary(evt.target.result);
        };
    }

This is my upload.php

$inputHandler = fopen('php://input', "r");
$loc = "uploads/" . $_GET["name"];
$fileHandler = fopen($loc, "w+");

while(true) {
    $buffer = fgets($inputHandler, 4096);



    if (strlen($buffer) == 0) {
        fclose($inputHandler);
        fclose($fileHandler);
        return true;
    }

    fwrite($fileHandler, $buffer);
}

My question is, how do I encrypt those upload file using AES or mcrypt while the file is in streaming mode?

Answers:

Answer

It was something like this. This is from memory and untested, because I don't have the PHPSecLib library on my Laptop, and I am too lazy to set that all up...

require __DIR__ . '/vendor/autoload.php';

use phpseclib\Crypt\AES;
use phpseclib\Crypt\Random;

AESStreamEncode($input, $output, $key)
{
    $cipher = new AES(AES::MODE_CBC);
    $cipher->setKey($key);

    $iv = Random::string($cipher->getBlockLength() >> 3);
    $cipher->setIV($iv);

    $base64_iv = rtrim(base64_encode($iv), '='); //22 chars

    fwrite($output, $base64_iv); //store the IV this is like a salt

    while(!feof($input)) {
        $contents = fread($input, 1000000); //number of bytes to encrypt 
        $encrypted = $cipher->encrypt($contents);
        //trim the = or ==, and replace with :, write to output stream.
        fwrite($output, rtrim(base64_encode($encrypted), '=').':'); 
    }
}

AESStreamDecode($input, $output, $key)
{
    $cipher = new AES(AES::MODE_CBC);
    $cipher->setKey($key);

    $buffer = '';
    $iv = false;

    while(!feof($input)) {
        $char = fgetc($input); //get a single char
        if($char ==':'){
            if(!$iv){
                $iv = base64_decode(substr($buffer, 0, 22).'=');  //iv is the first 22 of the first chunk.
                $cipher->setIV($iv);
                $buffer = substr($buffer, 22); //remove the iv
            }
            $buffer = base64_decode($buffer.'='); //decode base64 to bin
            $decrypted = $cipher->decrypt($buffer);
            fwrite($output, $decrypted);

            $buffer = ''; //clear buffer.
        }else{
            $buffer .= $char;
        }
    }
}

Where $input and $output are valid resource stream handles like from fopen etc.

 $input = fopen($filepath, 'r');
 $output = fopen($ohter_filepath, 'w');

 AESStreamEncode($input, $output, $key);

This lets you use things like php://output as the stream if downloading the decrypted file.

You have to remove the = because it is sometimes missing or 2 of them, So we cant rely on them as a separator. I usually just put 1 back on and it always decodes it correctly. I think it's just some padding anyway.

References

PHPSecLib on GitHub

PHPSecLib Examples

The encrypted file should look something like this:

xUg8L3AatsbvsGUaHLg6uYUDIpqv0xnZsimumv7j:zBzWUn3xqBt+k1XP0KmWoU8lyfFh1ege:nJzxnYF51VeMRZEeQDRl8:

But with longer chunks. The IV is like a salt and it's pretty common practice to just add it to the front or back of the encrypted string. So for example

[xUg8L3AatsbvsGU]aHLg6uYUDIpqv0xnZsimumv7j:

The part in the [] is the IV, (its 22 chars long after base64_encode) I counted it many times and it always comes out that long. We only need to record the IV and set it one time. I suppose you could do a different IV for each chunk, but whatever.

If you do use PHPSecLib, it also has some nice sFTP stuff in it. Just make sure to get the 2.0 version. Basically it has some fallbacks and native PHP implementations for different encryption algos. So like it would try open_ssl then if you were missing it, it would use their native implementation. I use it for sFTP, so I already had it available. sFTP requires an extension ssh2_sftp and If I recall it was only available on Linux at the time we set things up.

UPDATE

For downloading you can just issue the headers then give the decode function the output stream, something like this

 $input = fopen('encrypted_file.txt', 'r');
 $output = fopen('php://output', 'w');

 header('Content-Type: "text/plain"');
 header('Content-Disposition: attachment; filename="decoded.txt"');

 header('Expires: 0');
 header('Cache-Control: must-revalidate, post-check=0, pre-check=0, max-age=0');
 header("Content-Transfer-Encoding: binary");
 header('Pragma: public');

 //header('Content-Length: '.$fileSize);  //unknown

 AESStreamDecode($input, $output, $key);

These are pretty standard headers. The only real catch is because the filesize is different when it's encryped you can't just simply get the size of the file and use that as it will be quite a bit bigger. Not passing the filesize won't prevent the download, it just wont have an estimated time etc.

But because we know the size before encrypting it, we could embed it in the file data itself like this:

 3555543|xUg8L3AatsbvsGUaHLg6uYUDIpqv0xnZsimumv7j:zBzWUn3xqBt+k1XP0KmWoU8lyfFh1ege:nJzxnYF51VeMRZEeQDRl8:

And then pull it out when we do the download, but you would have to use as separate function to get it and it might be a bit tricky to not mess up decoding the file.

Honestly I think it's more hassle then it's worth.

UPDATE2

Anyway, I worked up these changes for embedding the file size, it's an option, but it could also mess up the decryption of the file if not done carefully. (I haven't tested this)

AESStreamEncode($input, $output, $key, $filesize = false)
{
    $cipher = new AES(AES::MODE_CBC);
    $cipher->setKey($key);

    $iv = Random::string($cipher->getBlockLength() >> 3);
    $cipher->setIV($iv);

    $base64_iv = rtrim(base64_encode($iv), '='); //22 chars

    //Option1 - optional filesize
    if(false !== $filesize){
        //add filesize if given in the arguments
        fwrite($output, $filesize.'|');
    }

    /*
        //Option2: using fstat, remove '$filesize = false' from the arguments
        $stat = fstat($input);
        fwrite($output, $stat['size'].'|');
    */

    fwrite($output, $base64_iv); //store the IV this is like a salt

    while(!feof($input)) {
        $contents = fread($input, 1000000); //number of bytes to encrypt 
        $encrypted = $cipher->encrypt($contents);
        //trim the = or ==, and replace with :, write to output stream.
        fwrite($output, rtrim(base64_encode($encrypted), '=').':'); 
    }
}

So now we should have the filesize 3045345|asdaeASE:AEREA etc. Then we can pull it back out when decrypting.

AESStreamDecode($input, $output, $key)
{
    $cipher = new AES(AES::MODE_CBC);
    $cipher->setKey($key);

    $buffer = '';
    $iv = false;
    $filesize = null;

    while(!feof($input)) {
        $char = fgetc($input); //get a single char
        if($char =='|'){
            /*
              get the filesize from the file,
              this is a fallback method, so it wont affect the file if
              we don't pull it out with the other function (see below)
            */
            $filesize = $buffer;
            $buffer = '';
        }elseif($char ==':'){
            if(!$iv){
                $iv = base64_decode(substr($buffer, 0, 22).'=');  //iv is the first 22 of the first chunk.
                $cipher->setIV($iv);
                $buffer = substr($buffer, 22); //remove the iv
            }
            $buffer = base64_decode($buffer.'='); //decode base64 to bin
            $decrypted = $cipher->decrypt($buffer);
            fwrite($output, $decrypted);

            $buffer = ''; //clear buffer.
        }else{
            $buffer .= $char;
        }
    }
    //when we do a download we don't want to wait for this
    return $filesize;
}

The decode get filesize part acts as a fallback, or if you don't need it then you don't have to worry about it messing the file up when decoding it. When downloading we can use the following function, that way we don't have to wait for the file to be completely read to get the size (this is basically the same as what we did above).

//We have to use a separate function because
//we can't wait tell reading is complete to 
//return the filesize, it defeats the purpose
AESStreamGetSize($input){
    $buffer = '';
    //PHP_INT_MAX (maximum allowed integer) is 19 chars long
    //so by putting a limit of 20 in we can short cut reading
    //if we can't find the filesize
    $limit = 20;
    $i; //simple counter.
    while(!feof($input)) {
        $char = fgetc($input); //get a single char
        if($char =='|'){
            return $buffer;
        }elseif($i >= $limit){
            break;
        }
        $buffer .= $char;
        ++$i; //increment how many chars we have read
    }
    return false;
}

Then when downloading you just need to make a few changes.

$input = fopen('encrypted_file.txt', 'r');
//output streams dumps it directly to output, lets us handle larger files
$output = fopen('php://output', 'w');
//other headers go here

if(false !== ($filesize = AESStreamGetSize($input))){
    header('Content-Length: '.$fileSize);  //unknown
    //because it's a file pointer we can take advantage of that
    //and the decode function will start where the getSize left off.
    // or you could rewind it because of the fallback we have.
    AESStreamDecode($input, $output, $key);
}else{
    //if we can't find the filesize, then we can fallback to download without it
    //in this case we need to rewind the file
    rewind($input);
    AESStreamDecode($input, $output, $key);
}

If you want to shorten this you can just do it this way too, it's only about 19 chars at most so it's not to big a performance issue.

 if(false !== ($filesize = AESStreamGetSize($input))) header('Content-Length: '.$fileSize);

 rewind($input);
 AESStreamDecode($input, $output, $key);

Basically above, we just do the filesize header (or not) and then rewind and do the download. It will re-read the filesize, but that's pretty trivial.

For reference fstat(), Hopefully that makes sense.

Tags

Recent Questions

Top Questions

Home Tags Terms of Service Privacy Policy DMCA Contact Us

©2020 All rights reserved.