5 interesting technical challenges I faced when building FilePond
FilePond is an MIT licensed JavaScript file upload library released in March 2018. It’s available as a native JavaScript plugin but can also be used with Vue, React, Angular, and jQuery using special adapter components.
We’re going to take a look at the FilePond animation engine, the way it renders the drop area, how the image preview plugin leverages the new createImageBitmap API and lastly we’ll look into the limitations of our good old friend the file input.
Animation Engine
I wanted to make a central animation loop drive the entire FilePond interface as it can become very tricky to create high-performance animations with multiple loops while also reading information from the DOM.
The FilePond interface consists of a tree of views, each of these views has a root element and has a read and write function. For every tick of the core animation loop, FilePond calls the read method of the root view, which then calls the read methods of its child views, and so on. Then, when all views have been read, it calls the write method on the root view and this view calls the write methods on its child views, and so on till the end of the view tree is reached. So it basically does a read on the FilePond DOM subtree and then a write on that same subtree.
The read method, reads layout information from the DOM. This contains data like the current element position, width, height and margins. The retrieved layout information is then stored in each view so it can be used in the write method. Because all reads are grouped the browser only has to do layout calculations once, subsequent requests for layout information are “free”.
The write method is in charge of updating the view. It will apply the layout information and update text elements and apply animations attached to the view. These animations can be tweens or spring based animations.
The core animation loop runs till no more actions are received, views have been updated and all animations have been completed and the interface reaches an idle state. FilePond then waits till the idle state is disturbed by either the user or an API call.
Because the DOM reads and writes have been grouped and we only animate using `transform` and `opacity` there’s no layout trashing (write methods can’t invalidate layout as reads have already finished), the browser only has to do compositing, resulting in very fast performance.
Animating the File Drop Area
The FilePond drop area needed to grow vertically with each file dropped. I badly wanted to animate this as all other elements are animated as well and having the drop area size change instantly felt very out of place.
- I could use
scaleYto scale the drop area on the GPU but scaling a rounded rectangle will cause its rounded corners to stretch, which looks weird. - Or I could animate the
heightproperty but that would be very slow as it can’t be composited on the GPU. Also, it doesn’t do subpixels so as you can see in the animation below when the left square is near max height, the animation becomes less smooth.
To circumvent this we can use a technique called 9-Slice Scaling
Instead of one div, we use three separate divs for drop area.
- A static top div that renders the top-left and top-right rounded corner.
- A middle div that is scaled over the y-axis.
- A bottom div that is translated and renders the bottom-left and bottom-right corner.
To simulate the height animation we set the middle div height to 1px and use the scaleY() transform to change its height. Setting transform: scaleY(100) will result in a 100px high div. At the same time, we can move the bottom div to a 100 pixel vertical offset using transform: translateY(100px)
By combining both transforms we can create the illusion of animating height, the animation performs well, our corners stay nice and sharp and we get the smoothness of subpixel positioning.
Fake Progress Indicator
On a fast connection, a tiny file can be uploaded to the server in the blink of an eye. This might cause your users to wonder if their files have actually been uploaded as the status almost immediately switches to“upload complete” skipping over the “busy uploading…” state.
To prevent this uncertainty, FilePond will show a fake progress indicator for the first second of each upload. If the upload takes longer the progress of the actual file upload takes over. This might give the user additional confidence that the file has actually been uploaded.
Try for yourself, which version feels better?
What’s up with the File Input?
It’s very unfortunate but apart from serious style issues, our beloved file input has another more pressing limitation. It’s impossible to set a file input’s files property. I tried, vigorously.
I fully understand the reason why setting a path to the value attribute is all but secure (you could point it to a file on the user’s file system), but having the option to add a newly created File object to the files list would be very useful.
At the moment files that are drag-n-dropped have to be uploaded asynchronously. As the File objects are contained in the drop event they can’t be stored in the file input. They either have to be kept in memory or have to be uploaded using XMLHttpRequest (or fetch) at once.
The same goes for File objects created in the browser. For instance, files generated with a text editor, or images that have been modified on the client.
If we could set or add files to the files property of the file input.
- It would be easier to set up form validation, as both dropped files and files selected on the file system via the browse window can be stored in the same list.
- We would no longer have to modify the server to accept asynchronous uploads. Often we’ll want to know which files have been uploaded so we can reference them in the final form submit.
- We could easier apply progressive enhancement techniques. The server will only receive a list of files, no matter where they originated from. If a certain CMS exposes a file input in its form module, we could do whatever we want on the front-end, as long as we set the files list.
To circumvent this problem the FilePond file encode plugin can encode files as base64 strings. To do this without stalling animations files are sent to a Worker thread for encoding. When the worker is done encoding, a base64 encoded string is sent back and stored in a hidden input field.
Once uploaded the base64 string can then be turned back into an actual file on the server.
While this creates the opportunity to send files synchronous along with the parent form post, this creates some other problems. One has already been stated, the data needs to be turned back into a file on the server. Another one has to do with memory usage. When submitted, strings take up a lot more memory than file objects. This causes some browsers to bug out when a form with lots of data is being submitted. Another thing to keep in mind is that some server security software will mark the form data as suspicious (cause of the length of the values).
Image Loading
When dropping an image on FilePond I wanted to show a preview, this adds a bit of color to an otherwise monotone experience. As previewing images should be optional I moved this functionality to a separate plugin. The plugin would then render previews of dropped images and adjusts these previews based on the crop information supplied by the image crop plugin.
Problem is, if you want to render a preview of a big JPEG encoded image, it’ll take the browser some time to decode the image and render the resulting Bitmap data.
While the browser is decoding, the main thread stalls and everything freezes. That’s bad news for any running animation.
To work around this problem, FilePond internally has an option to queue CPU heavy operations. It’ll await running these operations till it’s in idle state.This allows timing of heavy operations and prevents accidental freezing of animations.
I imagined it would still be nice to show some sort of busy state once the browser goes in lockdown. As it turns out, I was in luck, WebKit based browsers can run CSS animations even while the main thread is frozen. Before handling the heavy load queue FilePond will switch to a busy state and the file progress indicator will start its infinite spin animation (unfazed by the main thread that is about to be blocked).
Further speed improvements to the image preview plugin could be made by decoding images on a separate thread. We can do this with the recently added createImageBitmap API (at the time of this writing supported on Chrome and Firefox). The File object is sent to a Web Worker which uses createImageBitmap to decode the file and then returns a BitmapImage object which can be rendered to a <canvas/> using thedrawImage method.
That’s it for now!
Let me know if this was interesting! I’m happy to write some more about other challenges like rendering smooth gradients, making FilePond accessible for screen readers, handling and reading JPEG EXIF orientation and writing the adapter components for React, Vue, and Angular.
If you have any questions, find me on Twitter
