sclaggett

Native Drag and Drop and Large File Uploads with the Flash Player using HTML5, JavaScript, and ExternalInterface

I was challenged several months ago to get native drag and drop working with a browser-based Flex application. Basically, we wanted to be able to drag one or more files from the desktop and drop them onto the Flex application, but the previous developers tasked with the enhancement had declared it impossible. Much to the surprise of everyone on the team, what would have been trivial for an AIR application turned out to be much trickier in a web browser. Indeed, others online had also declared it impossible, but some clever users had noticed a way this might work in modern browsers and had gone as far as creating several proofs of the concept. The mechanism they described used HTML5 drag and drop events to detect when a file was dragged into the browser and dropped on the Flash player. The file contents were then loaded by the JavaScript drop event handler and passed to the Actionscript code using ExternalInterface.

Figure 1

I implemented the same mechanism in our application and it worked quite well. The Flex application would display the dropped files in a grid, allow the user to add metadata, and upload the files to a server when the user clicked the "Upload" button. This mechanism worked fine for small files, but it started to fall apart as the size of the files increased and due to the greater memory and processing requirements. The solution I landed on for handling large files was quite simple: cache the file reference in JavaScript, and upload it to the server asynchronously using XMLHttpRequest when prompted by the Flash player. This approach uses the native upload mechanism of the browser to avoid the need to load the entire file into memory at once.

Figure 2

There is a nice implementation of the drag and drop mechanism created by Wouter Verweirder, but I didn't come across his version until after writing my own. One thing I love about his version is the automatic injection of the necessary JavaScript functions by the Flash application, a technique that greatly simplifies migration from local development to a production framework. I've since updated my solution to use this approach as well. His implementation is well done and worth checking out, especially if you don't need to upload large files.

You can download the compiled SWC for use in your own application from here. Details on how to use the library are provided in the next section. Source code for a sample application that uses the library can be found here, and source for the library itself here. Everything has been tested and works as designed on Firefox 26.0, Chrome 32.0, Safari 6.1.1, and Internet Explorer 11.0.2.

The last four sections of this page go into the details of the inner workings of the library. Four aspects of the project are explored in more detail: initialization, drag and drop, browsing for files, and uploading files.

Using the library

Prepare the project. Download the SWC from the link above and save it in the libs directory of your project. Flash Builder should automatically detect the library and add it to the Reference Libraries.

Figure 3

Open index.template.html in the html-template directory with your favorite editor, add the parameter wmode to the code that embeds the player, and set it to opaque. This commonly needs to be done in more than one place if you're using Adobe's stock template so it will work in all major browsers.

Figure 4

It's important that you set the wmode parameter or drag and drop mechanism will not work.

Configure the drag and drop class. The drag and drop mechanism is implemented in a single static class named DragAndDrop. There are five settings that control the behavior of the drag and drop and upload mechanisms and dictate which features are enabled during initialization. The default settings should be sufficient for most use cases, so you may not need to explicitly set any configuration parameters.

  1. disableMainWindow – This flag specifies if drag and drop should be disabled at the level of the main browser window. It is generally desirable to disable this because the user may accidentally drop a file on the web page outside of the Flash player if your application does not occupy the full browser window. The default browser behavior when this happens is to redirect away from the HTML page that contains the application and load the dropped file in its place, which probably wasn't what the user intended to do. Default value is true.
  2. enableFileTransfer – Flag that specifies if the dropped file contents should be loaded into memory by the JavaScript code and passed to the Flash player. Set to false if your application does not need access to the file contents as this will improve performance. Default value is true.
  3. fileTransferThreshold – Threshold number of bytes above which the file contents will not be loaded into memory and passed to the Flash player. This value is only used if the enableFileTransfer flag is set to true. Be aware that larger files take longer to load and require more memory. Default value if 40 MB.
  4. enableFileBrowse – Flag that indicates if the Actionscript code will be able to open the native file selected dialog of the browser through a JavaScript function. Set this to false if this functionality is not needed. Default value is true.
  5. enableFileUpload – Flag that indicates if the Actionscript code will be able to upload files using the native mechanism of the browser. The only files eligible for upload by this mechanism are those the user selected by either drag and drop or using the native file browser. Set this to false if this functionality is not needed. Default value is true.

Add event listeners. The DragAndDrop class emits nine different events, each of which is an instance of the DragAndDropEvent class. Add listeners for the events you would like your application to receive:

  1. DRAG_ENTER – Dispatched when the user drags something over the Flash player.
  2. DRAG_LEAVE – Dispatched when the user drags something away from the Flash player without dropping.
  3. DROP_START – Dispatched when the one or more files have been dropped on the Flash player or selected using the file browser dialog. This is the first event to be emitted when files are being loaded and transferred to the player. The event includes the total number of files being processed.
  4. DROP_STOP – Dispatched when the file loading and transfer operation is complete.
  5. UPLOAD_START – Dispatched when the upload of one or more files is starting. The event includes the total number of files being uploaded.
  6. UPLOAD_PROGRESS – Dispatched when the browser updates the JavaScript regarding the status of an upload. The event includes the name of the file and the percentage of the upload that is complete.
  7. UPLOAD_FILE – Dispatched when the upload of a file is complete. The event includes the name of the file, server response, number of files uploaded, and total number of files in the upload operation. The latter two parameters are useful for displaying total upload progress to the user.
  8. UPLOAD_STOP – Dispatched when the upload of all files is complete.
  9. UPLOAD_ERROR – Dispatched when an error occurs during the upload of a file. The event includes the name of the file.

Set callback functions. There are two events in the lifecycle of the drag and drop control that are implemented as callbacks so the application can specify a return value synchronously.

The first of these is a callback function that will be invoked repeatedly as the user drags items over the Flash player. This function, set using the DragAndDrop.dragOverCallback parameter, gives the application the opportunity to do hit testing and set the mouse icon that the browser displays. It has the following signature:

function DragOverCallback(clientX:int, clientY:int):String

The clientX and clientY parameters are relative to the upper left corner of the stage. The callback should return one of the four drop effect constants defined in the DragAndDrop class: DROPEFFECT_NONE, DROPEFFECT_COPY, DROPEFFECT_LINK, or DROPEFFECT_MOVE. If a callback function is not specified then the control will default to DROPEFFECT_MOVE.

The second callback is a function that will be invoked once for every file the user drops on the Flash player or chooses using the native file selection dialog. This function, set using the DragAndDrop.dropFileCallback parameter, receives a DragAndDropFile object that contains information regarding the dropped file including the file contents if they have been loaded, the number of files that have been loaded, and the total number of files in the load operation. The latter two parameters are useful for displaying total progress to the user. The callback has the following signature:

function DropFileCallback(file:DragAndDropFile, filesLoaded:int, filesTotal:int):Boolean

The callback function should return true if it wants the JavaScript code to cache a reference to the file, a requirement for uploading the file later using the native browser mechanism. The default behavior if a callback function is not specified is to not cache file references.

Initialize. Once everything above is squared away, initialize the drag and drop control by calling DragAndDrop.initialize(). This function will return an empty string if successful or a description of the error otherwise. At this point, you’re ready to go! Simply drag and drop files and watch the events and callbacks fly. Refer the section below for a more detailed description of what happens internally during initialization.

Browse for files. Your application can open the native file selection dialog by calling DragAndDrop.browseForFiles(). Files that are selected will be loaded by the JavaScript code and transferred to your application in the same manner as those added by drag and drop. The reason to select files this way instead of using the normal Flash framework is to have a common mechanism for loading and uploading files regardless of whether they were selected by drag and drop or the browser dialog. The sample application shown below is opening the native file browser dialog in response to the user clicking a button in the Flash player:

Figure 5

See the section below for a more detailed description of how browsing for files works.

Upload files. File references that have been cached by the JavaScript code can be uploaded asynchronously using the native browser upload framework by calling DragAndDrop.uploadFiles(fileList:Array). The fileList parameter is an array of BrowserUploadFile objects, each of which contains the file name, target upload URL, and upload mechanism (e.g. POST). The JavaScript code will locate each cached file reference based on the name field, so it is important that this matches the original file name specified by the browser. See the section below for a more detailed description of how uploading files works.

Initialization

The first step in initialization is to inject the necessary JavaScript functions into the DOM. I first got acquainted with this technique when extending the functionality of FlexIFrame, so it was already familiar when I read Verweirder's code. However, his idea to use this technique here is a great one because of the simplification it provides to deployment. The functions injected by this project are:

The only downside to injecting JavaScript functions this way is it makes debugging more difficult, as most debugging tools (e.g. Firebug) don't seem to acknowledge the existence of functions added through such a mechanism.

Second, a number of Actionscript functions are exposed to the JavaScript code by registering them with ExternalInterface. These functions allow the JavaScript code to communicate drag, drop, and upload events to the Flash player:

The third step in initialization is to disable drag and drop operations on the main window of the web page, i.e. the background of the page outside of the Flash player. As described above, this is important if the Flash application doesn't occupy the entire browser window to prevent accidental navigation away from the HTML page that contains the SWF. The screenshot below demonstrates that the page surrounding the Flash player no longer functions as a drop target, but the application does:

Figure 6

Fourth, the control checks to make sure the browser supports the FileReader interface if file content loading is enabled so the file contents can be loaded and pass to the Flash player. The configuration parameters described above enable an application to load small files for preview without grinding the whole system to a halt if a large file is added.

Fifth, once everything has been injected and all checks have passed, the control invokes a JavaScript function that adds the drag and drop event listeners to the Flash player. The fact that ExternalInterface knows the ID of the Flash player object makes it easy for the JavaScript to locate the object.

Sixth, an HTML input control is injected so the browser's file selection dialog can be opened programmatically. This allows the application to load and upload files the same way regardless of whether there were added by drag and drop or through the file selection dialog.

Finally, an HTML form is injected to enable the application to use the browser's native mechanism to upload files to a server. The step is essential for transferring large files without the overhead of loading them into memory all at once.

Dragging and Dropping

A series of drag and drop events are detected by the JavaScript functions and passed to the Actionscript code through ExternalInterface. The first events to consider are drag enter and leave, each of which is only emitted once, and whose handlers invoke the functions asOnDragEnter and asOnDragLeave in the control. Drag enter is triggered when the user drags something over the Flash player, and drag leave is emitted when he drags it away without dropping. Security restrictions in the HTML specification prevent the JavaScript code from getting any significant information about any files the user may be dragging at this point.

The next event to consider is the drag over event. This event is emitted continuously while the user is dragging something over the Flash player. The JavaScript code calculates the coordinates of the mouse pointer relative to the upper left corner of the stage and passes them to the Actionscript function asOnDragOver to allow hit testing. The mouse pointer can be changed by the application by returning one of the drop effects defined in the DragAndDrop class: DROPEFFECT_NONE, DROPEFFECT_COPY, DROPEFFECT_LINK, or DROPEFFECT_MOVE. The sample application below illustrates using hit testing and the drop effect to visually indicate that the grid is a valid target but the rest of the application is not:

Figure 7

Like the drag enter and leave events, security restrictions prevent the application from getting any significant details regarding files the user may be dragging.

The JavaScript drop event handler first invokes the Actionscript function asOnDropStart to notify the application that one or more files will be transferred. The handler then loads each file using the FileReader interface if two conditions are met: the configuration settings allow for the loading of file contents and the file size is below the threshold. Regardless of whether the file contents are loaded, the Actionscript function asOnDropFile is invoked once for each file, either immediately or after the load completes. The parameters passed to this function contain the file name, size, type, modified date, contents (if loaded), the number of files that have been loaded, and the total number of files that are being loaded. It returns a Boolean value that indicates to the JavaScript if the reference to the file should be cached for upload at a later time. The function asOnDropStop is invoked once all files have been transferred to the application.

The functions asOnDropStart and asOnDropStop are useful because the drop operation will take a significant amount of time if the files are large and the contents are being loaded and transferred to the Flash player. An application should listen for the events emitted by these function and provide a visual indication that a transfer is in progress so the user is not frustrated by a UI which appears to hang. The file counts provided to the asOnDropFile function allows the application to inform the user of the progress of the drop operation:

Figure 8

Browsing for Files

A hidden HTML input is injecting into the DOM during initialization of the drag and drop control by the JavaScript function jsInjectBrowseInput if this feature is enabled. The format of the input tag is:

<input id='fileBrowser' type='file' multiple='true' style='display: none' />

This simple tag enables the Actionscript code to programmatically open the browser’s native file selection dialog by calling the JavaScript function jsBrowseForFiles. The same code that loads the files added by the drag and drop mechanism also handles files selected use the browser dialog, providing a common way to conveniently address two use cases without any extra effort on the part of the application developer.

Uploading Files

A hidden HTML form is injecting into the DOM during initialization of the drag and drop control by the JavaScript function jsInjectUploadForm in a mechanism similar to that used to browse for files. The format of the form tag is:

<form id='uploadForm' enctype='multipart/form-data' style='display: none;' />

The Actionscript code calls the JavaScript function jsUploadFiles with an array of files it would like uploaded to specific URLs. This JavaScript function locates the cached reference for each file and employs a combination of the hidden form and XMLHttpRequest to asynchronously upload each file to the corresponding target URL. The contents of the file are loaded by the browser framework and passed to the server in the filedata field that is added to the form dynamically. This mechanism allows large files to be read from the disk in a series of small chunks during upload and avoids the need to load the entire file into memory at once. JavaScript event listeners detect upload progress, completion, and error events and pass them to the Actionscript code where they can be presented to the user:

Figure 9