Building My Own Image Host

I am building my own image host on pickmy.org, when I say image host; I mean something a bit more hacked together than just putting images in a directory and turning autoindex on. But something that’s a cross between imgur and a simple self-hosted image gallery.

Here’s what I’ve been able to do, where I need to go from here, and why I’m doing it this way.

Imgur was nice for a while. I’ve uploaded my fair share of images there from both my PC and phone to easily share them. I did this over serving them just off http since…it was in essence easier. I didn’t have anything setup to just drag an image in to a browser for upload; nor did I have anything easy/automatic for image uploads from my mobile device. But the issue with imgur is it’s getting to be a pain to use; they keep changing the interface, moving things around, and the mobile app has become bloated and half useless. So I decided to see what I could hack together.

My main goal was to have something that was as easy to upload as imgur from both my desktop and my phone. There is no shortage of photo management packages out there for self-hosting. The problem is that most of them were too “big” for what I wanted or didn’t serve images the way I wanted. For example, I tried Piwigo and while it was a nice package; I didn’t like the way the images were not only original file name, but accessed through a .php file with a file path similar to WordPress. If I’m sharing images in chat; I don’t want a very long URL.

I looked over several other packages, Piwigo was the only one that didn’t look like it was going to involve a lot of hacking and coding; and if I was going to do that, I wanted something a bit cleaner and simpler. I’m not trying to impress anyone with a flashy dynamic imagehost. While WordPress is a CMS and can be made to do what I want; it’s just not suited for sharing of simple links. For example, the image below’s url is https://pickmy.org/wp-content/uploads/2020/08/linx-server.png – that’s a little complex and long.

Linx-Server

My search lead me to a small project called Linx-Server; and on the surface it does most of what I want it to do.

As you can see from the basic screenshot, it’s a pretty no-frills system. You can drop your images in the box or click to load a file dialog; you load the image and that’s pretty much it. It does feature file expiration in case you want the images to disappear after a specific amount of time. It also wasn’t very complex to get going as it’s largely one self-contained binary. But when you upload an image; the server immedately gives you a link to the image. It doesn’t automatically load anything like imgur; it just gives you a link.

It does, like imgur; provide a server generated page with the image in it. You do have the option to enable hotlinking if you wish. So if I modify the link from https://img.pickmy.org/v9v8zsrt.jpg to https://img.pickmy.org/i/v9v8zsrt.jpg we can get just the raw image.

That’s perfect! That’s exactly what I want. I can change that /s/ to whatever I want too…so I can make it /img, or /i, or /lookatthisshit if I want. But this isn’t going to be replacing the traditional way I do images in WordPress posts. It’s still easier for me to click “image” in the block editor and upload/embed that way. This is quite literally just for those “quick shares” for forums or chat rooms; although as you can see above, you’re perfectly capable of embedding images from there.

But this comes with some down-sides; linx-server is itself a self-contained application. I don’t actually write any code to integrate it. What it can do, however, is behind nginx using fastcgi_pass rather than it’s internal http(s). You do this by specifying a location directive in your server block to point to linx-server; which is running on localhost. If you’re like me, and you’re running it on the root of that domain; linx-server effectively hijacks that entire domain due to the location directive. Well, not really….there are ways around that. But why go through the hassle?

Linx-server doesn’t offer a gallery. It is quite literally share the image, give you the link, and that’s it. The server itself does not have a way of showing you what files are there unless you know about them. This really doesn’t pose too much of a problem for me in most cases; I can literally just shell in to the machine and look at the filenames the directory where they’re all stored.

As you can see, v9v8szrt.jpg is stored here, as well as another image I used for testing and a couple of index files. What I’ve done is directed linx-server to just store everything in the root I had setup for piwigo; so it stores the actual images and the metadata in two folders. Normally, you’d probably store both of these folders outside of the www root; but I’m not. That’s because, for starters, everything in /var/www gets backed up daily as written in other posts. I’m not 100% worried about preventing hot-linking; in fact part of my purpose is to have something that can be hotlinked.

But with the default configuration; you can’t access the /files folder directly. The location directive for linx-server looks like this:

location / {
        fastcgi_pass 127.0.0.1:8080;
        include fastcgi_params;
    }

In this configuration with linx-server running on the root of a domain; every request passed to that domain in turn gets passed to linx-server’s cgi. Of course, if you wanted to run linx-server on a different URI; then some of tese problems could be eliminated. I could move linx-server to img.pickmy.org/u/ for example; and then only URI’s requesting /u would get passed to it. That would also mean sharing of images by default would be img.pickmy.org/u/s/img.jpg – I could configure it so images come up as img.pickmy.org/a/n/img.jpg which could fit in with the theme of the domain name….but I’m kind of set on the /img/image.jpg as a lotlink.

Of course, if you haven’t figured out; it’s storing files in /files/ of the web-root (which I will likely change when I put it in to final production use), but hotlinked images are served out of /s/ (or whatever I configure it). Thats because, again, nginx’s location directive matches /s/ URI to linx-server’s configuration.

Let’s See If We Can Break

Aside from keeping linx-server’s storage in the www-root making it part of the dailuy backup; it also means I can easily access the files in nginx IF I tell it to. Since Linx-Server lacks an image gallery; then why not get something that creates an image gallery; and set it to serve out of linx-server storage? It’s just a folder in the web-root and after disabling cleanup in linx-server; it ignores the extra files in the storage; as noted from index.html and index.php being in /files already. But before I can think about this…I need to figure out if I can get nginx to not send URI requests to /files to linx-server, but I also need to make it continue to process PHP files.

   location  /files {
        index index.html index.htm index.php;
                location ~ \.php$ {
                include snippets/fastcgi-php.conf;
                fastcgi_pass unix:/var/run/php/php7.4-fpm.sock;
     }
    }

Here’s the version that works. I played around with this for several hours last night and got as far as getting the /files location directive to work; this took longer than normal since nginx.org is down and the good documentation isn’t available. But I had another problem…how to make it continue to process PHP files. I read and read and found nothing; then I got the crazy idea that maybe location directives could be nested. Afterall, after reading stuff about the nginx config it seemed like it might support that kind of thing; server and location blocks were made out to be discreet configurations that might over-ride other configurations. After googling for “nginx nested location directives”; the first page spoke around nested location and gave examples.

Adding the nested directive for PHP files within the /files directive solved my problem. Not only does nginx not send request to /files to linx-server, but it will also properly process the php files. This means I can add an external gallery package to serve an image gallery out of the main storage.

That’s about as far as I’ve gotten thus far. With that much working I can now start focusing on finding a gallery program; I didn’t want to start researching those before figuring out how I was going to serve it up.

Securing Uploads & Mobile Upload

Uploading images, at least on the back-end, seems to be completely handled by sending a PUT request to /upload/, the API documentation goes through exactly how this works. To prevent anyone from uploading; it uses an “API Key” that you send as part of your header. If you’re doing it from the browser, then it will give you a password prompt where you can type it in. So there is a basic level of security there, which is all I’m pretty requiring out of this. If someone finds the key and starts uploading junk; I’ll see that in the backup activity. But what about uploading from mobile? Well…that too was a complicated solution.

LinxShare For Android

So there is an Android application that supports uploading to your linx-server; however….it does not have a binary distribution. This means I’d have to google to see if someone has one….or build my own. How hard could this be? I mean…people do it; just because I have no clue where to start doesn’t mean that should stop me, right?

Right! Because I’m not ashamed to admit Google is my co-dev; I googled for how to build an APK from a source. I found a set of decent instructions for doing so; they were actually part of a medical-targeted application. With the way medical devices are regulated, you can’t actually provide pre-built versions; but it’s legal for you to build your own and use it.

So I created an 8 vCPU vitrual machine with 6 gigs of ram on the Xen Hypervisor here at home and threw lubuntu on it. I then followed the instructions on installing Android Studio to build the APK; except substituting the git repository I actually wanted to use. I feel sorry for people on slow internet that have to do that. Android Studio was an 800MB download on it’s own; and it acted like it downloaded another 800MB after it was installed. Here at home that’s not an issue, I have fiber optic internet; and the amount of stuff it had to do wasn’t an issue as it had 8 cores, plenty of ram, and a speedy RAID array for disk access.

So I built the APK and then fought with my phone’s security over it, then fought with GooglePlay security over it. It loaded in the end; and I was actually surprised when I punched in my server info, API key, and told it to upload an image. It actually worked! There’s no fancy GUI or anything with the app; you simply browse on your phone and share the image with the LinxShare app. It then uploads the image and copies the URL to your clipboard.

So…there we have it. This is how I’m building a shitty imghost.

Site comments disabled. Please use Twitter.