$100 for a simple script to sync files from one server to another - said a craigslist post.
It was a Friday, I wanted to verify my developer skills, so I reached out with a quick text message - only later seeing that all inquiries should have come through email.
As a developer who's done a few file transfers over the years, I figured I could write a quick rsync script and be done with it. $100 for thirty minutes of work could have been a fun, Friday project.
Developers in the room will know that when a non-technical person claims "simple" -- it's time to slow down.
It was a simple task. It just wasn't limited to the task he thought he was asking for. He didn't know.
The server where the files were located didn't allow direct ssh ( secure shell ) access. But, there was an html page with a list of all the .xml files that could be downloaded.
Okay, I'll just grab all the links.
Oops. There are too many files for the connection time allowed on the server. Will have to filter the list first.
Wait. This has to happen multiple times per day, because they sometimes update the files.
No problem. We can write a cron-script to repeat the work on a schedule.
As you can tell, this went from simple to not-so simple.
The non-developer didn't understand the constraints of the coding environment. Permissions weren't clear. Workarounds had to be considered. Cache had to be invalidated. Error checking had to be enabled to prevent input errors from crashing the server.
I plan to get the thing working again, but this time, I'm going to do it my way. Now that I understand the constraints, I'm going to control the logistics.
My plan is to filter the input and ensure I'm working with only the files that matter. Then, I'm going to put up a proxy server to pre-download the files and ensure direct access. I'll bypass the server timeout by doing incremental file sync and file diff checks to ensure I'm only getting changed files rather than copying all files even if unchanged.
Again, a developer with experience will be thinking, "yep."
Software makes things easy and simple. If you can get it working, it can stay working. Any changes to the inputs or the environment, however, can easily break code that didn't know to expect change.
So, the next time you think you've got an idea for an easy software project, think about how easy it seems to just eat your veggies and do a few push-ups each day.
It was a Friday, I wanted to verify my developer skills, so I reached out with a quick text message - only later seeing that all inquiries should have come through email.
As a developer who's done a few file transfers over the years, I figured I could write a quick rsync script and be done with it. $100 for thirty minutes of work could have been a fun, Friday project.
Developers in the room will know that when a non-technical person claims "simple" -- it's time to slow down.
It was a simple task. It just wasn't limited to the task he thought he was asking for. He didn't know.
The server where the files were located didn't allow direct ssh ( secure shell ) access. But, there was an html page with a list of all the .xml files that could be downloaded.
Okay, I'll just grab all the links.
Oops. There are too many files for the connection time allowed on the server. Will have to filter the list first.
Wait. This has to happen multiple times per day, because they sometimes update the files.
No problem. We can write a cron-script to repeat the work on a schedule.
As you can tell, this went from simple to not-so simple.
The non-developer didn't understand the constraints of the coding environment. Permissions weren't clear. Workarounds had to be considered. Cache had to be invalidated. Error checking had to be enabled to prevent input errors from crashing the server.
I plan to get the thing working again, but this time, I'm going to do it my way. Now that I understand the constraints, I'm going to control the logistics.
My plan is to filter the input and ensure I'm working with only the files that matter. Then, I'm going to put up a proxy server to pre-download the files and ensure direct access. I'll bypass the server timeout by doing incremental file sync and file diff checks to ensure I'm only getting changed files rather than copying all files even if unchanged.
Again, a developer with experience will be thinking, "yep."
Software makes things easy and simple. If you can get it working, it can stay working. Any changes to the inputs or the environment, however, can easily break code that didn't know to expect change.
So, the next time you think you've got an idea for an easy software project, think about how easy it seems to just eat your veggies and do a few push-ups each day.
I had an epiphany about software just a month ago when I put two and two together.
For awhile I've been studying Taylor Otwell's code for a product that was intended to become Laravel Forge Pro. After having the entire backend completed (with tests) he decided it wasn't revolutionary enough of an idea and pivoted to create Vapor instead to provide Laravel a serverless option.
But when you read that code it's nothing special. Every line and every file is just plain. And that's what makes it brilliant. Good software engineers write boring fucking code. It's the bad ones who write the impressive functions that you would see as a StackOverflow answer that dazzles newcomers.
The moment I put two and two together was when Taylor was being asked in an interview -- can't remember which one -- what his software was like. And he plainly just chuckled and said something like I don't know why everyone wants to see what my code is like. My programs are just like any basic Laravel App. Nothing special.
And then I was like. Awh. That's why it's so good.