Exotic Security

Migrating to GitHub Pages

2021-08-05T08:00:00+08:00

Over the last few years I have not been updating this blog, it's for all the usual reasons to do with time and commitments. Along with that, I'd gotten pretty lax about patching the server it was running on. It had Debian unattended upgrades switched on, but really I should have been taking more care of it.

I think it's irresponsible to leave an unmaintained server running on the internet and so over the last weekend I've migrated everything to GitHub Pages. I've been using Pelican to generate HTML from markdown for the site so uploading that into a git repo was pretty seamless.

It does mean I can't run this site as a .onion site any more, that was a fun experiment while it lasted but again it's not something that I was activity maintaining.

How to mount a VHD file on linux

2018-11-15T08:00:00+08:00

How to do it

Install libguestfs

For Debian and Ubuntu this is pretty easy

sudo apt-get install libguestfs-tools

For Red Hat, Centos, Fedora

sudo yum install libguestfs-tools

For Arch Linux (my distro of choice) you need to install 3 packages from the AUR

git clone https://aur.archlinux.org/hivex.git
git clone https://aur.archlinux.org/perl-sys-virt.git
git clone https://aur.archlinux.org/libguestfs.git

cd hivex
makepkg -si
cd ../perl-sys-virt
makepkg -si
cd ../libguestfs
makepkg -si

Once you have libguestfs as a normal user (not root / sudo) run guestmount

guestmount --add old_server_backup.vhd --ro /mnt/vhd/ -m /dev/sda1

--add option is for the image you want to access
--ro sets to read only, alternatively you could use --rw for read / write
/mnt/vhd the path where you want to mount the drive
-m /dev/sda1 specify which partition within the .vhd file you want to mount.

Enjoy :-)

My Rant

So I recently had to mount a backup of an old VM that was saved as a .vhd file and so I googled "How to mount a VHD on Linux" and the first result that came up was a Ubuntu forumns post where someone had asked the question and the top reply was someone else telling that person to go off and use a search engine. Followed by a link to Stack Overflow which didn't actually answer the question.

Some support forums and Stack Overflow in particular can be quite toxic to new comers (and even toxic to experienced veterans) and it's very infuriating when the top result on google is someone being told to just search google for the answer.

I know it can be annoying when simple questions come up over an over again but at the very least try to link to a useful article and if possible quote the relevant bit.

Anyway I hope my instructions saved someone the frustration that I went through.

Wordfence reivew

2018-11-08T08:00:00+08:00

TLDR: I tried the free version, I like it.

Amongst other things I do in my day job, I administer a WordPress site. We had a security audit and one of the findings was that our site was misconfigured to show a different failure message for a login when the username exists to when it doesn't. This allows for user enumeration which makes brute-forcing easier because you don't waste time trying to brute force accounts which don't exist.

My first thought was that the auditors had made a mistake, we were running very vanilla WordPress on the latest version and I thought surely that's something the WordPress team would have patched if it was an issue with the default install.

It turns out I was wrong, I couldn't find any definitive statement from the WordPress team but it seems they don't think user enumeration is an issue¹. Along with different log in prompts, there are several other places in WordPress that leak usernames such as appending ?author=1 to the URL of the site.

So I went looking for a way looking for a way to patch that and found Wordfence.

After installing it I checked the password failure message and that was fixed. Then I started looking through some of the other features and was impressed with the brute force protection, they have sensible defaults and fairly good metrics.

I also saw they had a scanner which checks the integrity of the WordPress core files which is a good idea.

The plugin can also provide two-factor authentication which is a great idea but that's a paid feature.

Over all, I'd say it's a good plugin and I will be installing it on any WordPress sites I'm responsible for in the future.

To be clear, this is not an unreasonable view to hold. It's more of an information disclosure than a real security threat. It really depends on what type of site you are running as to how serious this is. ↩

The right to repair

2018-11-01T08:00:00+08:00

The Self Repari Manifesto by iFixit

Recently a mate of mine blogged about how Macbooks can only be repaired by apple technicians. This is a story that sort of passed me by but it's becoming increasing common.

I wouldn't have got into IT or learnt anywhere near as much as I have if I hadn't been able to tinker with things. I'm a tinkerer by nature and that's how I learn.

It seems many companies now days are going out of their way to make it hard to open up their product and understand how things works. This isn't just Macbooks, it's everything from lawnmowers to electronic doorbells.

My predictions about autonomous vehicles

2018-10-25T08:00:00+08:00

No one can predict the future, so I'm going to try. Maybe it will be fun to come back to this post in 10 years and see how laughably wrong I was.

Traffic will become less hectic, not more

CGP Grey posted a great video The Simple Solution to Traffic which shows cars weaving in and out at rates that would only be possible with self-driving cars that all know what everyone else will do.

But I don't believe we will ever get to this level of automation. Or at least not for an incredibly long time, I see three problems standing in the way of this:

Cooperation - Will Ford cars talk to Toyota and Tesla talk to Chevy?
Security - Let's say we get cross-vendor communication going, how do we secure that communication so you don't get bored hackers jamming up traffic.
Other things on the road - The biggest blockers I see is other stuff on the road. That person who still wants to drive their vintage Ford Model T and even if we passed a law to ban all non-autonomous vehicles, what about cars that have broken down or that little kid who ran onto the road chasing their ball.

From what I've read current autonomous cars feel like being in a car with an overly cautious driver and I expect that trend to continue.

Private ownership of vehicles will end

This isn't really my theory, I stole it from a talk by Paul Fenwick.

If you buy a self-driving car and it drops you off at work in the morning, why pay for parking in the city? why even drive it home and park it there? Why not put it to work doing taxi rides while you're at work? as long as it's back to pick you up in the afternoon you might as well make a profit from it while you're not using it.

But then if you extend that if your car is spending more time working as a taxi than being used by you, why not always rent a corporate-owned car when you need it and not need to deal with maintenance and insurance and such.

Cars will chain together like trucks with trailers

Achieving something between a bus and a car I can imagine cars that have tow bars on the front and the back and will link up. If four cars are going in the same general route why not link them until they need to split up, it saves on fuel. It might not quite achieve the crazy traffic mentioned above but would act much as trucks and buses do in current traffic.

My kids will never learn to driver

I don't have kids yet, but I am optimistic about timelines and I'd like to believe I'll be able to take an autonomous taxi ride within the next 10 years.

Paid parking lots will be a thing of the past

I guess this might be an obvious one, but why pay for parking at work or at the airport when you can just get the car to drop you off and drive away.

The internet reaction threshold

2018-10-18T08:00:00+08:00

I've noticed a lot of the sites I hang out on are getting more an more vitriolic, there are lots of theories about why this is and I'd like to add the Internet Reaction Threshold to the mix.

Say someone posts a controversial video to YouTube. I like one by Matt Gray and Tom Scott where Tom presents the idea that lettuce is just used as a cheap filler vegetable as a wildly controversial view and the later in the video says "kids are basically small little tornados of destruction" as though it's just an accepted fact.

Let's go with the lettuce thing although if it helps you feel free to mentally replace "lettuce" with anything that's genuinely controversial, guns, abortion, systemd, politics, whatever.

Say someone posts a video about lettuce, 95% of the people watching will either agree or disagree but can't be bothered adding their comment. It takes a lot of effort to write "Sure, I prefer spinach, but lettuce is cheaper and easier to grow, I guess it's all down to what your willing to pay for, let's all get along." and if people don't care that much they just go off and watch the next cat video.

Only the people with really strong feelings will take the time to comment, and so the comment thread becomes polarised.

The Internet Reaction Threshold coupled with the fact that posts which get the most reactions are promoted creates a horrible feedback loop. People who post neutral feelings about lettuce don't get as much attention as those who are super-pro-lettuce or super-anti-lettuce.

Graph made with xkcdgraphs.

What is yak shaving?

2018-10-11T08:00:00+08:00

I know things like the Jargon File exist, but I'm having fun writing this.

There is a gif from Malcolm in the Middle that's been floating around the internet for years that explains Yak Shaving better than I ever could.

It's basically when you need a series of often trivial things before you can do your main goal. The way it was explained to me was:

Your friend asks you to restore a WordPress site onto their server, it should take 5 minutes to do so you say sure.

You want to restore WordPress from a backup
but to do that you need to update their PHP
but to do that you need to fix a broken dependency on an old PHP package
but to do that you need a library that has to be compiled from source
but to do that you need to install a GCC
but to do that you need to fix an issue with their apt-get
but to do that you need to schedule a reboot
but to do that you need to ...
...
...
but to do that you need to hike to the top of a Tibetan mountain and shave the hair off a Yak.

What is bikeshedding?

2018-10-04T08:00:00+08:00

If you have spent much time hanging out on technical email lists or forums you might have come across the term "bikesheding" essentially it means to spend time talking about the little details which everyone has an opinion on and to some degree ignoring the important things.

It comes from a book called Parkinson's Law: Or the Pursuit of Progress which is responsible for many economic theories but in particular the Law of triviality which talks about a committee approving a massive nuclear power plant and none of them really understand nuclear reactors so they approve it without comment.

But when it comes time to build the bike shed at the power plant for the employees to park their bikes everyone has an opinion on what material it should be made from, wood, fiberglass, tin and what colour the shed should be painted and so that gets discussed for hours.

Recently one of the technologists I admire most, Guido van Rossum, the inventor of Python stepped down after a change to the Python language that was made particularly difficult by the fact that everyone had an opinion. It was ultimately a fairly small change, but very controversial.

I've been in meetings about websites I'm working on where I've thought, I don't really like that font, or I don't really like that particular shade of red. But I've held my tongue because I know that if I start the conversation, everyone will have an opinion about the colour scheme. The thing is, as long as the colours are not so bad that they will cause accessibility issues, it's far more important to focus on content and functionality.

People will ignore warnings that were wrong in the past.

2018-09-27T08:00:00+08:00

This is basically a boy who cried wolf story.

I was recently on a road trip through Newfoundland in Canada, and as we were driving along. I was driving and I saw a sign that said construction ahead. Followed by a sign warning that the speed limit would soon change to 50 kph. That was closely followed by a sign that said up to $1,500 fine for speeding in a construction area.

I didn't want to get a $1,500 fine so I dutifully slowed down from 100 kph to 50 ... Yes, I was that guy with a string of cars behind me. After about 2 or 3 km with not a single sign of construction, I passed a "construction ends" sign and speed up again.

We changed drivers a few times along the trip and went through several more "construction zones" which didn't have the slightest sign of construction and eventually began to just ignore them like everyone else was doing.

Then sometime after the 30th zone we suddenly came across a bunch of people working on the road and had to brake sharply because we weren't expecting them.

It's the same with things like SSL warning messages, I've lost count of the number of times I've seen SSL warning messages when connecting to wireless networks which have captive portals. I know that it's not because they are trying to man-in-the-middle my banking details, but they are trying to redirect me to their site so I can agree to their terms and conditions.

Accepting that captive portals are a thing that won't go away, Android and Firefox on Linux have a solution where they try to reach out known site over HTTP and if the connection gets redirected then they pop up with a little warning message that says "This network requires a sign in..." and takes the user to the sign in page.

I assume other OS and Browsers have similar features. While I wish captive portals didn't exist, I also accept they are not going away. A little "sign in" pop up is better than training people to click through SSL warnings whenever they connect to an open wireless network.

The one-time pad is not a perfect cipher

2018-09-20T08:00:00+08:00

A little bit of knowledge is a dangerous thing.

This will come as no surprise to professional cryptographers but it's a mistake that I see armature cryptographers make over and over again.

When you do crypto 101 you learn that one-time pad provides "Perfect Secrecy" and that it's provably secure, it's mathematically impossible to break it.

But while the one time pad provides perfect secrecy it does not provide integrity. It is therefore vulnerable to a known plaintext attack.

If you know what the message is, you can change it without the change being detected. Consider the following scenario;

In the army Alice has a messenger boy called Malroy who she suspects of being a spy, she wants to send the message to Bob, the General of the army, but she can't trust her messenger. So she writes the message "Execute Malroy Immediately" encrypts it with the one time pad she has shared with Bob and hands Malroy his own (encrypted) death warrant.

As it happens Alice was right, Malroy was a spy and as it happens he knows what the message says and decides he wants to change it to "Promote Malroy Immediately"

Malroy can simply xor "Execute Malroy Immediately" with "Promote Malroy Immediately" and then xor that with the encrypted message to change its continence.

That might seem like a contrived example, and to a degree it is. But known plaintext attacks are a real problem and crop up more often than you might expect. There was a recent attack on LTE which was using AES-CTR which also doesn't provide authenticated encryption. The cryptographers figured out where in the packet the IP address of the DNS server was and they could inject the IP address of their own DNS server without breaking the encryption, and then use their own DNS server to get man-in-the-middle access to phones.

A simple hack to get around VPN IP conflicts

2018-09-13T08:00:00+08:00

sudo ip route add 192.168.1.2 dev ppp0

Recently my Aunty had some problems with her backup software at her business and asked if I could help. I could connect in to their VPN but their network was on the 192.168.1.1/24 subnet and I was on a hotel WiFi that was also on the 192.168.1.1/24 subnet.

The best solution would be to change the subnets, but I couldn't really ask the hotel to change their network and my Aunty wasn't about to change either.

There are some ways you can make it work with DNS and NAT but that's a lot of work to set up and I really just wanted to access one server quickly.

Instead in Linux you can use the ip command to tell it that you want to route packets via a specific interface so I was able to force all packets for 192.168.1.2 over the VPN.

sudo ip route add 192.168.1.2 dev ppp0

I was able to access the server and fix the backups. It's not a good long term solution because it will break other things, but it's a neat little hack that can get you out of a pinch.

Rediscovering F-Droid

2018-09-06T08:00:00+08:00

I recently bought a Nokia 6.1 and I've been Loving it but it's running stock Android and one of the best ways to secure your Android device is to not install random apps that you don't trust, and especially not to install untrusted apps that didn't come from the Google Play store.

On my old Samsung Galaxy S4, I ran Lineage OS (previously CyanogenMod). On my Nexus 6P it was a mix, so on my Nokia 6.1, I tried to just use to Play Store for a while. However, I trust the F-Droid store and I wanted a couple of apps that are not in the Play Store so I installed it.

I'm not sure when Android made the change¹ but now instead of having a single "Allow unknown apps" switch in the settings, there is one per App, so when I downloaded the F-Droid APK I was asked to allow Firefox to install unknown apps.

Then after that, I was asked to allow F-Droid to install untrusted apps. Once I'd allowed that I went back and removed the permission from Firefox:

Long press on Firefox icon > App Info (Drag it to the top) > Install unknown apps (scroll to the bottom)

Now I've got the F-Droid store on there I'm much happier, they have updated the UI and it's much cleaner, but more importantly, the F-Droid store just has much less horrible junk apps.

Very, very few of the Apps in the F-Droid store contain advertisements, and fewer still contain in-app purchases.

The official Google Play store is filled with ad-supported apps and it provides almost no way to filter them, for example, I'd love a search function where you could restrict it to only Apps that don't have ads and don't push in-app purchases.

A classic example is when you search for "Flashlight" the top result that comes up contains both ads, and in-app purchases.

and on top of that, it requires access to:

Location
- approximate location (network-based)
- precise location (GPS and network-based)
Photos / Media / Files
- read the contents of your USB storage
Storage
- read the contents of your USB storage
Camera
- take pictures and videos
Other
- view network connections
- full network access
- control vibration

That's a lot of permissions for something that's just going to switch the LED light on and off.

As a counterexample when you search "Flashlight" in the F-Droid the first result that comes up has no ads or in-app purchases and only needs camera² and flashlight permissions which seems reasonable.

For a while I toyed with the idea of creating a web crawler and indexing the Play Store to create a site which provides advanced search of apps on the Play Store to filter out the junk but that seems like a lot of work to build and maintain for very little gain when the F-Droid store contains a great repository of nicely curated open source apps.

I suspect it was between Android 7.0 Nougat and Android 8.0 Oreo. ↩
From what I can tell the camera permission is needed because some builds of Android won't allow access to the LED without camera access. ↩

Making #FFFFFF brighter

2018-08-30T08:00:00+08:00

Years ago I used to occasionally read a site called "Clients from hell" basically a page where people (generally freelance graphic designers) could post funny stories about horrible clients and misunderstandings.

Generally, the stories went along the lines of:
Client: I want you to do X work for me.
Freelance graphic designer: Sure that will cost Y dollars.
Client: Oh, I didn't think I'd have to pay you for that, it's just pictures and stuff.

or the ever popular misunderstandings of how technology works
Freelance web designer: I've updated the site with the changes you wanted.
Client: Really? Because the paper copy I printed out last week hasn't changed.

However there was one story that came up over and over that always annoyed me, it runs along the lines of;
Client: Can you make X thing brighter?
Freelance graphic designer: No, it's #FFFFFF, it's impossible, it is as bright as it can be.
Client: Please, just a little brighter would make it look better.
Freelance graphic designer: changes nothing ... Sure there you go, it's brighter now.
Client: Ok, thanks.
Freelance graphic designer: Posts on Clients from Hell about how dumb the client is.*

The thing is, it's not the client that's at fault there, it's the designer.

I'm sure many people would have seen the checker shadow illusion before.

for those of you that haven't, the squares A and B are the same colour. This can be verified by using your favourite image editing tool or even with Firefox press F12, select the eyedropper tool and you will see they are both #787877

When the client is saying "Make it brighter" what they probably mean is "Make it appear brighter" and as a graphic designer, it's their job to explain that maybe we can make the background darker and that will make the foreground look brighter.

It's the same in IT security, our job as professionals to interpret what the client is asking for. When a client asks for "A website that's impossible to hack" as a security professional, you can't guarantee there will never be any 0 day in Apache or Nginx or whatever you use. But you can explain to the client that maybe what they are really looking for is a static website hosted on GitHub or whatever is appropriate for their situation.

Nokia 6.1 Review

2018-08-23T08:00:00+08:00

TLDR

I bought a Nokia 6.1, I've used it for 3 months, I think it's brilliant. My wife's old phone needed replacing, I was so happy with my phone that I suggested she get a Nokia 6.1 too.

Some context

In May 2018 I bought a Nokia 6.1 because my Nexus 6P died. I managed to get the 64GB storage / 4GB RAM / Dual Sim version for just $300 AUD. So it's a mid-range priced phone, it's less than a third of the price of the latest iPhone or Google Pixel which each retail for over $1,000 AUD in Australia.

The thing is despite being a technology enthusiast who runs Arch Linux to get the latest and greatest I'm actually quite minimalist. Both on my phone and my laptop I only install the packages I need and despise bloat.

As such I wanted to get a phone that ran Android One so I would get as close to the stock Android experience as possible without all the vendor bloat. I remember when I got my Samsung Galaxy S4, it came from Telstra (Australia's largest telco) and came pre-installed with about 30 junk apps like the "AFL Footie scores" or the "Samsung Store" and "Crayon Physics" which could not be removed until I wiped the whole phone and put on CyanogenMod.

Similarly, as a security professional, I wanted a phone that was guaranteed to receive regular security updates for at least two years¹. Again that meant a phone that was in the Android One program.

Lastly with new phones that come out each year, for the last 5 years or more there has really been nothing other than a Fingerprint reader that I've thought was a huge improvement.

Every year it's the same black glass brick with no buttons. Maybe this year's model has rounded corners and last year's had square, or maybe this year's phone's CPU is 5~10% faster or has a little more ram, or has slightly more pixels, slightly better battery life or whatever. But it's always just a small improvement over last year's phone. In fact, storage on most phones went down for a while, I had a Nokia N81 in 2007 with 8GB of storage. Later I got a Nokia N97 which had 32GB storage with an SD card slot which is better than many phones sold today.

I don't play fancy games that need great 3D graphics or lots of CPU/RAM.

What I'm saying is that I'm very happy with a low-end phone.

The actual review

In for what I need the Nokia 6.1 is if anything overkill. It's very snappy, all the apps open quickly, I've never noticed any lag, the one game I do play on my phone Clash of Clans run smoothly without a hitch.

The camera could be a little better in low light conditions but it's still pretty good. Below are a few pictures taken on the Nokia 6.1, I have not added any filters or post-processing to correct the colours. Click on them to see the full picture with no metadata removed.²

In fairness, I've picked some of the best pictures, there were a lot of bad ones too, but it shows what can be done

Physically it feels very solid. Some of the low-end phones can feel very plasticy (and there was some story about bending iPhones) but the Nokia 6.1 feels great. I've dropped my phone on cement and stone a few times (without a case) and it hasn't even scratched yet.

The battery life is good, constant heavy uses playing games and watch movies on a plane with the screen on full brightness it lasts about 12~13 hours, if left in my pocket and just used to take the occasional photo or send WhatsApp messages I can easily go 48 hours or more on a single charge.

The fingerprint reader is good but not quite as good as the one on my old Nexus 6P, occasionally I've needed to try a few times to unlock the phone especially with sweaty hands.

From a software and security point of view, it's outstanding. As I've mentioned above it is part of the Android One program. Which means it gets security patches over the air every month and stays on the latest patch level. So far I've got the patches within a week or so of them being released.

I only have two complaints one is that the battery is not replaceable so when it inevitably wears out I will need to get a whole new phone.

The other is that I can't easily unlock the bootloader and reflash my phone which for 99% of people is not a problem anyway.

Conclusion

After I'd had my phone for about 2 and a half months my Wife's phone (Sony Xperia Z5) started showing dead pixels and we decided it needed replacing. I was so happy with my Nokia 6.1 and my wife had used it a fair bit and liked it too so we decided to get a second one for her.

Actually I think that there should be a law that phone manufacturers must support the devices they sell until a given date, see my rant on Expiry dates on smart phones and other IoT devices. ↩
I know this includes things like the GPS coordinates where the photo was taken, I'm fine with that. ↩

Live chat support on websites

2018-08-16T08:00:00+08:00

I'll just come out and say it, I like those live chat support things on websites.

I feel like I should dislike them because they are usually clunky and often show as a popup at an inconvenient time rather than just an option on the contact us page. There are stories about websites getting compromised because the 3rd party JavaScript they added to get the chat tool has been compromised.

But despite all their failings, I still like them. I think it's because when I need support on a website I don't want to send an email because I want support now I don't want to wait hours for each response.

I also don't like to pick up the phone and call. Maybe I'm unique in that but;

Calls are ephemeral and I like to have a record of what was said.
I can't multi-task well while I'm on the phone. I can open a chat window then go to another tab and periodically check to see if I've got a reply (or wait for it to make a noise or whatever). If I got a reply 45 seconds ago, the support person is not going to mind that it took a minute for my reply. But when I'm on hold on the phone I feel like I need to be constantly attentive because if the support person says "Thanks for holding" and I don't reply for 45 seconds they might just hang up.
There are no issues with accent, often people have issues understanding my Australian accent and I have issues understanding their accent. I've learned the phonetic alphabet and that helps but it's still a struggle sometimes.
I can think about my reply and re-read what they said. This is kind of a mix of the two points above but I feel it's still distinct. If someone says "Do you have xyz reference number" I can spend a minute to look for it (probably copy and paste it from wherever it is) and send it back without all that "Ummm... yes, I've got it here somewhere just let me rummage through my papers... ahh... here it is just let me read it out for you"
I can use Google Translate. I've needed to contact an airline about a ticket and their support only spoke Spanish, it needed a bit of back and forth so chat worked better than email and I could just copy paste each of the messages into Google Translate. It wasn't perfect but it was pretty good and it got the job done.

Maybe I also like them because of the fact that as a teenager I spent several hours every night hanging out with my friends on MSN Messenger / IRC / Yahoo Chat / etc... so I'm just more comfortable with online chat.

How many android patterns are there?

2018-08-09T08:00:00+08:00

This is a post I've had kicking around in my drafts folder for just over 2 years now so I've decided to publish it as a partly complete problem.

One of my favorite pastimes, when I'm bored, is solving the puzzles on Project Euler¹. I'm not very far through but I've solved 56 at the time of this writing. It's as much about writing the code and learning the language, in my case Python, as it is about actually solving the problems.

The questions are good because they look like there must be a simple way to calculate the answer but it's not immediately obvious². An interesting question that I think is worthy of Project Euler is:

How many possible Android lock screen patterns are there? And how would you calculate it for arbitrarily sized grids?

Let's examine the standard size first, initially if we are just working with a 3 x 3 grid it might help to think of the positions as numbers from 1 to 9.

Initially, we might (incorrectly) think, there are 9 possible starting positions, then 8 remaining moves, then 7 and so on. So it would be 9!

>>> import math
>>> math.factorial(9)
362880

But then we can't use a pattern of fewer than 3 positions so if we remove all there digit options

>>> math.factorial(9) - (9 * 8 * 7)
362376

However this is not right for a few reasons, first 9! (362880) is only the number of combinations of length 9 so to get all the possible combinations

>>> import itertools
>>> positions = [1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> len(list(itertools.permutations(positions, 9))) # 362880
>>> len(list(itertools.permutations(positions, 8))) # 362880
>>> len(list(itertools.permutations(positions, 7))) # 181440
>>> len(list(itertools.permutations(positions, 6))) # 60480
>>> len(list(itertools.permutations(positions, 5))) # 15120
>>> len(list(itertools.permutations(positions, 4))) # 3024
# Total 985824

Second now we know how many combinations there are, we see that not all combinations are valid, for example while we can have 1234.

We can't have 1324 because there is no way to get from 1 to 3 without going through 2, even if you try to avoid it the line snaps to any positions it passes through.

I found a few incorrect solutions online which simply had a list of invalid moves such as from 1 to 3, from 7 to 9 and so on, but this is not correct either. We can't simply say that moving from 1 to 3 is always invalid because once a position has been used we can jump over it so we can have 2413 as a valid pattern which does go from 1 to 3.

This might be obvious, but just to clearly state it; While you can't jump over an unchecked position, you don't need to move to an adjacent position, for example, knights moves are valid, so we can have 1834

But just when we think we are getting a handle on things, LineageOS (previously CyanogenMod) throws a spanner in the works by allowing grids up to 6 x 6. For a larger grid, I think it's easier to switch to a coordinate system instead of numbered positions.

This brings in a whole new range of moves, for example [(0,3), (5,0), (2,5), (2,4), (2,3), (2,2), (2,1), (2,0), (5,5), (0,2)]

and it brings some new invalid moves, we can't go from [(0,0), (4,2)] without passing through (2,1)

After banging my head on a wall for a while, I searched online for a solution and the best answer I found was a 3 x 3 grid has 389112 possible patterns.

That's great, but every single correct solution I could find involved a brute force approach. Trying every possible combination and then discarding the invalid ones.

When it's just a simple 3 x 3 grid with only 985824 combinations to check brute force is not a bad way to go.

With a 4 x 4 grid (16 positions, over 4,000,000,000,000 combinations to check) brute force becomes incredibly hard but still within the realms of modern computers. By the time we get to 6 x 6 grids (36 positions, more than 2^128 combinations to check) it's downright impossible on current hardware.

There are some things we can do to speed things up though, for example the last two lengths (e.g. on the 3 x 3 grid that combinations of length 8 and 9) will always have the same number of possible combinations because every combination of 8 positions has exactly one corresponding combination of 9 positions.

So the problem that I haven't been able to crack is, can we design an efficient algorithm that can calculate the number of possible moves on an arbitrarily sized grid? not just square grids, what about 3 x 9 for example.

All pictures generated with Lock Pattern Generator

If you are a maths genius and you have a solution please get in touch. I'd love to know and I'll update this post with a link to your solution, michael at hybr dot id dot au

I know what you must be thinking, and you're right! I am great fun to sit next to at parties... Only I don't go out that much because who would want to be out with friends when you could be at home quietly solving maths problems!? All jokes aside I do enjoy Project Euler, it's like people who do sudoku or crossword to relax. ↩
Or at least it's not immediately obvious to me. ↩

The value of instant feedback

2018-08-02T08:00:00+08:00

I am a huge believer in the value of instant feedback within security. It's important to pick at what point you give feedback because you don't want to risk spamming users. It's been shown several times that if you show users warnings and they are regularly false alarms that people will tune out and ignore warnings.

But given at the right time, and not too often, giving instant feedback to users on what they are doing can provide great security controls.

Two examples of this are; user logins and important transactions.

If you have ever used Duo Push or Google's "Google Sign-In for Android" whenever you try to log in you will get a message on your phone saying, "Is it you trying to sign in?" this more than just 2 Factor authentication. You can get 2 Factor from any TOTP app like Google Authenticator, but this also lets you know if someone has tried to log into your account.

At work, all Administrators had two separate Active Directory logins, one administrative account, and a regular account. Most of the work could be done with just the regular account but if you ever logged in to a server with a domain admin account you would receive an email instantly. It didn't provide 2 Factor, but it gave feedback so if an account was compromised we would know about it.

Another great example was I have a Citibank credit card, and with the Citibank app, I can get push notifications every time there is a transaction. I think it's a great feature, sure someone could still use my card fraudulently once but I'd get a notification and contact the bank straight away rather than waiting until I check my statement.

Like I said at the top of the post, it's important not to spam people, but done right push notifications are a great security tool.

Don't put jokes in warnings

2018-07-26T08:00:00+08:00

There is a vlog series I quite enjoy Matt and Tom's park bench and one thing they have discussed a couple of times is that the London underground has a number of signs that are used for important information and warnings like "The Piccadilly line is not running today".

But sometimes, particularity around holidays like Easter, Christmas, and New Years they put up jokes on the signs like "We wanted to make a joke about Easter but couldn't think of one that was very bunny."

For someone who is fluent in English that might just be mildly annoying because you look up, see there is a notice on the LCD screen, worry that your train has been delayed, wait for it to scroll across before realising it's just a lame pun and then move on. For someone who is not fluent in English, it just adds to an already stressful situation.

The same should apply to UI design and security systems. I love a bit of humor in life, and some projects get this right. Python, for example, has a great mix of humor and in-jokes in their documentation to stop it from being too dry. But they don't put jokes into the important parts of the documentation or in error messages because the last thing some poor newbie who is debugging wants to read is some witty jokes that don't help them fix the problem.

It really shouldn't have to be said but:

When designing a system, don't put jokes in warnings or in functional bits of documentation.

Two new iPhone security features

2018-07-19T08:00:00+08:00

I am at heart a bit of an open source hippy who sees the world through rose coloured glasses. I want to believe that Android¹ is the best mobile operating system because it is at it's core open source and it gives users the freedom to run their own code and inspect the code running on their device without needing to pay licensing agreements or sign NDAs. Unlike the walled garden that Apple has built.

However, if pushed I'd begrudgingly admit that if security is your priority Apple and the iPhone have the edge.

Apple has released two new security features for their latest version of iOS and I think they are really interesting to look at from the point of view of threat modeling because they cater to almost polar opposites of the threat landscape.

The two features are USB Restricted mode and "tools for generating strong passwords, storing them in the iCloud keychain, and automatically entering them into Safari and iOS apps across all of a user's devices." these two features nicely demonstrate what Bruce Schneier called "Going Dark" vs. a "Golden Age of Surveillance"

With USB restricted mode, if someone has an iPhone and they are picked up by police, the police can no longer access their phone to look for incriminating material². This is a great step forward and has privacy advocates cheering, but it's not something that most people have to worry about.

While with the password generation an storage, this is something that will result in a greatly increased security for a huge number of people. People generally are bad at picking passwords, bad at storing passwords and absolutely terrible at not repeating the same password across numerous systems. Letting your iPhone generate a password for you and syncing that across all your Apple devices is going to be a hell of a lot better than what most people are currently doing. It's not only going to be more secure but it's going to be easier and people will always take the easy option, it's like LastPass but built into the operating system.

If I had to give anyone the job of securing all that data I'd say companies like Apple and Google are about as good as you can get. But if Apple are syncing the passwords across devices then they must be storing the passwords in such a way that it's possible to recover the cleartext. That just opens up a whole can of worms, even Apple's best developers are still human, and humans make mistakes or can be bribed or threatened. There was a story no that long ago where some celebrity had their iCloud account password guessed and compromising photos leaked online, and that was a result of Apple forgetting to rate limit one of their services, so these things do happen.

So TLDR

USB Restricted mode Good for the ~1% of people who are political activist or being surveilled by intelligence agencies? yes. Good for the ~99% of people who just want to keep their Facebook account safe? it can't hurt I guess.
Password generation and cloud storage: Good for the ~1% of people who are political activist or being surveilled by intelligence agencies? maybe, it depends. Good for the ~99% of people who just want to keep their Facebook account safe? Yes.

Actually, I still want to believe that in the future Maemo 5 from my old Nokia N900 or KDE Plasma Mobile will dominate the mobile market, but I'm at least somewhat in touch with reality, so Android it is. ↩
I feel like I'm flogging a dead horse here and I shouldn't need to say this but, just because something is incriminating or illegal doesn't mean it's immoral. There are still countries where it's illegal to be homosexual, a man could have pictures of him and his boyfriend kissing. Illegal yes, immoral no. ↩

Creating Guest WiFi passwords

2018-07-12T08:00:00+08:00

This is a half-baked idea that's been knocking around in my mind for a couple of years now, I've never implemented it and maybe it's more of a solution in search of a problem, but I digress.

I've seen several places where they have guest WiFi and they have had all sorts of strange solutions for making sure their guest WiFi is only used by guests and not all and sundry. Often this boils down to some horrible man-in-the-middle proxy that asks you for some sort of detail like your hotel reservation code or your flight number or some number printed on your coffee receipt.

So I thought about using a Time-based One-time Password to generate a new code every day. The same way that the codes in Google Authenticator (or you 2FA app of choice) but instead of a 6 digit code numeric with a period of 30 seconds use a 12 digit base64 code and a period of 24 hours.

Then use some sort of network management software (or python and requests cludged together) to set the code as the PSK for your Guest WiFi. And put that code up on an LCD screen or somewhere your customers can see it but not public.

I did say this idea was half-baked, didn't I? To be honest the idea just interests me more because I think it would be an interesting application of ToTP codes to generate new random passwords every day rather than because I can think of any situations where it would actually be practical and useful.

Why I've gone off bitcoin a bit

2018-07-05T08:00:00+08:00

I first heard about Bitcoin at a Perth Linux Users Group talk in 2012 at a Perth Linux Users Group talk, where I thought it was a great idea but didn't think it would really take off so I didn't really pursue it.

But it kept coming up again an again so finally I took a more serious interest. I still think it's a great idea but lately, I've gone a little off bitcoin for 3 reasons:

Power Consumption
Over-valuation
Over-Hyped block-chain technology

Power Consumption Power Consumption is an interesting one, I was listening to a Risky Business podcast about a year or two ago (I can't find the episode) and they discussed how much power was being used to run all the GPUs / ASICs that are mining bitcoin. They linked to an article which said bitcoin minding was using some ridiculous amount of power like 1.21 Gigawatts their comments were (paraphrasing) "I think the numbers are a bit rubbery but the general idea is right."

To mine bitcoin, you need compute power, and for that you need electricity, and for that people are burning fossil fuels.

Over-valuation There was a Last Week Tonight episode where John Oliver called cryptocurrencies a giant ponzi scheme. I'd love to get angry and defensive about that, but unfortunately, for the most part, he is right.

Most buyers of bitcoin are buying it as an investment, and that's really not the point of a currency. Sure, some people trade in currency for a living, but for most people, you would only by Vietnamese Dong if you were going to go to Vietnam and expected to buy something with the currency.

With bitcoin, people buy it to hold on to it because the price is going up, but the price is only going up because people are buying it as an investment. I'm not sure if the bubble will burst or what will happen in the future, but right now most bitcoin is not being used to facilitate trade.¹

I'd love for this to change, but I don't know if it will

Over-Hyped block-chain technology I feel like people will invest in anything that got the word block-chain in it right now, Extra Credits even did an episode "Can Blockchain Technology be a Game Mechanic?" and they talked about how they could use the blockchain to track a weapon throughout the game. They say:

"Okay, picture this: You start a game using a simple iron sword, you progress and level, and eventually use it to slay a boss named "Grillmig The Orc". Suddenly your simple iron sword becomes "Grillmig's bane", and with the name changed, it gains some extra stats.

Eventually, like all gear, you'll outgrow it and decide to sell it. Some upcoming player buys it and as they level they farm a ton of ghouls. And now it becomes: "Ghoul Slayer, the bane of Grill Mig". It gains more stats, but eventually this player too out grows it.

The weapon passes from player to player each time accruing new heroic associated with it, or unlocking achievements that no single player could ever do alone. Eventually, it becomes one of the most coveted swords in the game because its unique.

And any player who examines it can see the name of all the players who ever wielded it and what deeds they did. And if the designers were really clever, the sword would also benefit from having been used by characters that later did heroic deeds.

So when your character slays that final raid boss god dragon of nightmares, all of the sudden your first training sword no matter who currently has it, levels up and becomes the heirloom of the Great .

That's Awesome, and it's something that blockchain lets us easily do. "

Now, first of all, that's an amazing game mechanic and a really cool idea, but one of the top comments said:

Why does Blockchain make those MMO weapon concepts easier than any other methods? Would that not work with a standard database on a server?

And that was my thoughts exactly. A blockchain solves the problem where you have a distributed system the users don't trust one another but want to build trust into their system. But the key word there is distributed if you already have a central authority that everyone trusts then a blockchain simply becomes a lot of work for no gain. If you trust the game company running the servers to keep an accurate history of the scores you don't need a distributed blockchain.

And in fact, if you have a central authority it solves one of the "problems" with a blockchain which is that you can undo a transaction if it's malicious or accidental. What if in the example above someone found an exploit in the game and managed to kill all the bosses in the game at once, now their weapon has that stat and if you had a distributed ledger, you can't undo.

People are human, they make mistakes, having an undo button is a great thing.

Another way I've seen blockchain over-hyped and used in the wrong way was some article² about a bank that was going to start using it for tracking their transactions. It was still at the R&D stage and probably just a puff piece with no substance. But the idea was it would be used by one bank, by them self, internally, tracking transactions but if it's just one bank why bother?

They can always go back and edit their internal blockchain to change some historical transaction and then just recalculate all the future transactions again after that. Who is going to stop them? Sure it might take some extra compute power to catch back up but they can do it. If their blockchain was running on 3 servers before just get an extra 300 servers onto it until you catch up again.

There are existing cryptographic tools that would do a better job with significantly less overhead in that situation, like time stamps. Make a file with all your transactions, calculate a SHA256 sum of the file and get it timestamped by an external trusted authority. It would give you a ledger that's much harder to alter.

The blockchain was a genius idea. It solves a very specific problem, but it's often being applied in situations where it's not appropriate. It makes me think of a kid who uses a VPN to log into Facebook and then make a derogatory comment about a teacher and later says "But I thought a VPN would make me anonymous"

All that's not to say that I don't like bitcoin, I do like it, I just think it's worth looking at the bad as well as the good.

There are obviously some counterexamples to this, one of my friends has some bitcoin that he is using pay for things like a VPS directly with bitcoin. ↩
I can't remember the exact article I was reading but just Google "Bank blockchain" and you will find hundreds of similar ones. ↩

The best long haul flight tip I've got, bring an empty bottle

2018-06-28T08:00:00+08:00

An IT Security focused blog is not really the right place to give out travel tips, but it's my blog so here we go.

I've received lots of great travel tips over the years but one of the best ones was for any flight you go on, especially long haul flights was:

You can bring an empty bottle with you through customs and then fill it up and drink plenty of water on the flight.

There are many bad things about long flight, but I feel like most of these things are exasperated by being dehydrated. Airlines give out small amount of water in tiny cups infrequently. Also I think¹ that the air-conditioning in planes is set to a very low humidity which drys my skin.

Even through you can't bring more than 100ml of liquids through customs you can bring an empty water bottle and most airports will have water fountains (usually located near the toilets) that you can use to refill your bottle.

Filling up my trusty water bottle at Leonardo da Vinci–Fiumicino Airport, Rome, Italy

I know that should be obvious that you can bring an empty bottles through but I've seen a lot of people throw out bottles at customs rather than just emptying them. Alternatively you could just buy a bottle of water after passing through customs but if your travelling a lot that's a lot of wasted bottles and do you really want to pay $5 for a small bottle of water?

I know nothing about the air-conditioning settings of planes, or it's effect on skin, it's purely anecdotal, and could be completely wrong. ↩

Are loot boxes gambling?

2018-06-21T08:00:00+08:00

First of all, always read the disclaimer¹. I was watching an episode of Extra Credits a while ago and they were addressing loot boxes in games. At 4:22 in the video the question of whether loot boxes are gambling comes up, they say that more research is required but that the only study they could find found that

Both legally and psychologically, there's an important distinction between gambling and non gambling and that is the ability to cash out. Because you can't take your rare Overwatch skin and sell it back to Blizzard for actual spending money, the experience affects us differently. Video game loot boxes are less like craps and roulette and are more akin to a crane game, or a blind box, or the raffle for prizes at the county fair.

And I take issue with that², because I think that the way we define currency needs to change. The Oxford English Dictionary defines currency as:

A system of money in general use in a particular country.

But I think a better definition would be something along the lines of

Anything that is used to facilitation trade

In fact Extra credits did a brilliant series on The History of Paper Money where they discus things like large rocks that were used as currency.

All sorts of things have been used as currencies, there are stories of cigarettes being used as currency after World War II, even by people who don't smoke. Consider bitcoin, I would argue that bitcoin and other cryptocurrencies are currencies.

If you search online you will find plenty of stories of the gold in World of Warcraft (WoW gold) being worth more than some or other countries currency³, and people who make full time jobs farming WoW gold and then selling online to change it back to their local currency.

You could argue that WoW gold is not a "real" currency because you can only buy things in Wold of Warcraft with it, and to buy things outside you need to convert it into another currency, but I would say that is true to some degrees of all currencies.

Even if you take what I assume⁴ to be the most widely recognised and accepted currency, the US Dollar, you still can't use it to buy anything anywhere without converting it first. If you walk into McDonald's, the most American store I can think of, in Australia and try to buy a Big Mac with US Dollars you will be politely asked to go away and come back with real money. There is no currency that is universally accepted everywhere without the need to exchange it into another currency.

So, I believe that currencies in games, in particular games which allow players to trade with each other and therefore create an easy market for people to sell their in game items are "real" currencies. And by extension I believe extension loot boxes are gambling because you can cash out.

I am so not qualified to talk about this, I'm not a psychologist, I'm not a finance expert, I'm not even a game designer. Information Technology is my area of expertise, and specifically IT Security. However, this is the internet, I have opinions and a blog so here we are. ↩
To be clear here, I don't take issue with Extra Credits for citing that study, they are pretty up front about the fact that it's an area that needs much more in depth research and at least they tried to use a scientific study rather than just going on gut feel. ↩
For example from a quick Google search I found that 1 USD buys 6,300 WoW Gold, and currently 1 USD gets 22,700 Vietnamese Dong (VND). You would be hard pressed to find anything 1 VND, in fact a cheap Bánh Mì is about 10,000 VND. So I guess you could say a Bánh Mì in Vietnam costs about 3,000 WoW gold. ↩
This is completely anecdotal. ↩

Where in the world is Michael Van Delft?

2017-12-28T07:00:00+08:00

In August 2015, I set myself the goal of publishing one post per week on this site for at least a year. I've managed that goal, this will be my 124th post and so far I've had great fun writing.

By the time this post gets published I will have been married to my beautiful fiancée and have left Perth for our honeymoon an almost 12-month long trip backpacking around the world.

I have no idea if I will continue to update this blog, or just leave it. Although I suspect that if I do continue with it, my posts will not be as regular.

Here's to a good life.

Simple Windows SMTP relay

2017-12-21T07:00:00+08:00

In a recent post I mentioned that I had survived an Office 365 migration with only minimal scarring. We run HPE Content Manager which is "Enterprise Software". Just like the large government agencies that Content Manager is designed for it's big, slow, resistant to change, expensive, bureaucratic but yet incredibly despite all it's failings, it's occasionally capable of achieving great things like landing on the moon.

Anyway HPE Content Manager hasn't gotten around to implementing TLS before authentication for it's mail processing yet so it can't talk to office 365. So I was looking for a way to setup a mail relay but didn't want to setup a whole new VM just to relay mail.

I was surprised to find that Windows offers a mail relay built in to IIS.

Installing

Go to server manager and select Manage > Add Roles and Features

Skip past the before you begin page

Pick "Role-based or Feature-based installation"

Select the local server

Add the "SMTP Server"

This will also install IIS 6.0

Confirm the setting and install

Settings

Once the SMTP Server is installed open IIS 6.0. If you have a website on your server (such as HPE Content Manager Web Client) you will see two versions of IIS.

Right click on the SMTP Virtual Server and go to Properties

Under the Access tab select Authentication.

On the Authentication window, check that Anonymous access is available

Next from the Access Tab select the Connections window and ensure that only the IP address you want can connect.

Then from the Access Tab select the Relay window and again ensure that only the IP address you want will be allowed.

Then go to the delivery tab, we are going to need the three buttons across the bottom.

Under Outbound Security enter the user name and password and tick TLS encryption.

Under Outbound Connections change the port to 587.

Finally under advanced set the smart host to SMTP.office365.com

Point Mail to the relay

Now you can point HPE Content Manager or whatever it is that you need to relay mail for, to your server.

The importance of open standards

2017-12-14T07:00:00+08:00

I recently looked through my archives and was surprised to find that I hadn't blogged about this before as it's something I get quite passionate about.

Many companies try to set up proprietary standards in IT in an attempt to control the market. It's a horrible practice and needs to be stopped. Imagine if you bought a HP laptop, and then you wanted to connect a printer to it, and instead of a USB port, you needed to buy a printer with a specific HP-Connection. And that type of connector was different from a Dell connection, or Lenovo, or Toshiba, or Sony, or Acer, or ...

Having one standard type of connection benefits everyone. It makes life easier for the consumers and makes the IT industry as a whole develop faster.

A great example of this is things like Firewire vs USB. Many people would argue that Firewire was a better design but there were several patent issues as well as some weird copyright issues around the name leading to some companies calling it i.LINK, Lynx, or the generic IEEE 1394. Ultimately Firewire did not take off.

It shocks me that even though the EU passed a law that all phones sold in Europ must have a Micro USB (or later USB-C) charger, Apple blatantly flaunts this issue and continues to sell phones with their own proprietary connector.

I've lost count of the number of times people have asked me if I've got a phone charger. I've got Micro USB and USB-C and that will fit any phone made in the last 8 years, except the iPhone. It's not because I don't like Apple, it's because Apple refuses to support open standards. People wouldn't be as surprised if I didn't have the charger for a Nokia N-Gage, or an Ericsson T28 they would accept that it's an unusual phone and I can't hold every type of charger.

The next iPhone whatever version number that might be, would be no less great a phone if it was to come with USB-C.

Embrace open standard, it makes life easier for everyone.

Hang Gliding Over Hell, 3 drives die in a 6 drive NAS

2017-12-07T07:00:00+08:00

A while ago I wrote about learning from failure. This is a story of failure; Hardware failure, failure of design and failure of my self (The systems administrator) to not correct the issues earlier. It's hard to write about, but I believe that stories of failure can teach us just as much, if not more, than stories of successes.

I inherited a system where a number of VMs were running on top of two Hyper-V hosts, with a single NAS hosting a .VHD file that was shared as an iSCSI target for the storage of the .VHDs that were the VM disks.

Now, this design is not great for a number of reasons that will become apparent throughout this post, but a few that should immediately jump out is that the one NAS provides a single point of failure, and that using a .VHD file as the iSCSI target will not provide great performance, it would be better to use the disk directly.

So the first failure is that I never upgraded this setup to something more robust, I had ample time and the budget was available but I took an attitude of "If it's not broken, don't fix it". That is the wrong attitude to have, sure no one was complaining about the performance and it was running ok so but I could have fixed all of these issues before they even started, and I did not.

2017-03-01 03:00 Drive in Bay 1 dies

I'm blissfully asleep, maybe I rolled over and scratch my ear.

2017-03-01 03:00 Servers blue screen

In theory, because the NAS has a RAID5 + Hot Spare, everything should keep ticking along even after one drive dies. However the server froze up for just long enough the iSCSI connection timed out and the servers could no longer read and write to their disks, so all the VMs crashed.

Critically this includes the Exchange server, so the email alert about a failed disk that the HP ILO should have sent goes nowhere.

2017-03-01 08:30 I get to work, and reboot servers

I get to work and am immediately told that nothing is working, so I remote into the Hyper-V servers and reboot all the VMs. They come back up ok and I spend a bit of time trying to work out why they failed but somehow I completely miss the fact that one of the disks in the NAS has died. I think I assumed it was related to a bad Windows Update for Hyper-V or something.

2017-03-01 21:00 Servers blue screen again

The servers bluescreen again, due to read/write timeout. Only this time the nightly backups fail to run as well.

2017-03-02 08:30 I replace drive in bay 1

I get to work and for a second day, all the servers are down. I reboot them and then I finally I see that the drive in bay 1 of the NAS is dead. We don't have a cold spare on site so I go down to the local computer shop and buy a new drive.

Fortunately, It's a RAID 5 with Hot Spare array so it has already rebuilt into the hot spare.

2017-03-02 09:45 Rebuilding at 12% Hot spare dies. Rebuilding starts again at 0%.

I'm sitting at my desk watching the array rebuild onto the drive in Bay 1 and then suddenly... The hot spare dies. I suspect that the hot spare had actually been dodgy for a while, but because we were using RAID 5 with a Hot Spare rather than RAID 6 the disk wasn't in use and so we never got alerted that it was bad.

Fortunately, we still have enough data on the 4 remaining disks to rebuild the array but now instead of a straight copy from the hot spare to the new disk, it's got to actually calculate the parity bits all over again. So the rebuild is not running much slower.

I replace the hot spare with a new blank drive.

2017-03-02 20:00 Rebuilding at 80%. Drive in bay 3 marked "predicted failure".

I spend the rest of the day not being very productive and checking the rebuild progress every 10 minutes or so. Then at about 20:00 when I log in to have a look, the drive in bay 3 is marked as predicted failure.

Last night's backups failed, tonight's backups have not run yet because I've disabled them while the array is rebuilding. But if Disk 3 dies before Disk 1 is online we won't have enough disks left to rebuild the array.

2017-03-02 21:30 Rebuilding hits 100%

The last few minutes were nail-bitingly intense. But finally, the array has rebuilt. I start the backups and go to bed, unfortunately, the backups fail again.

2017-03-03 10:00 A consultant comes in

Where I work I'm essentially a one-person IT team, I've got a colleague who is very technical and good to bounce ideas off, but they are not in a full time IT role. But we have an external IT consultant that we use if I need help or to cover for me while I'm away on leave, so we called the consultant in for a bit of extra support.

2017-03-03 12:00 We remove the drive in bay 3. NAS crashes. We replace the drive in bay 3.

After some discussion, we decide that the next step is to remove the drive in bay 3. Rather than waiting until the end of the day, or for the predicted failure, let's just pull it and get the array rebuilding onto the hot spare as soon as possible.

I've pulled drives out of servers with hardware RAID before and they have been fine. I'd just recently pulled drives out of this NAS before and had no issues. But as soon as I pulled the drive out from bay 3, the NAS bluescreened and wouldn't reboot despite the fact that the OS was on a separate RAID array.

Naturally, all the VMs bluescreened as well and people were displeased that all the systems had stopped working in the middle of the day. I replaced the faulty drive back into bay 3 and then NAS booted.

We decided not to touch the NAS again during working hours.

2017-03-03 17:45 I start manually copying VHD files off to USB drive.

At this point, it's a Friday afternoon and I don't have a successful backup since Tuesday night. Veeam is failing, so I start copying the VHD files (for the VMs) onto a USB hard drive.

2017-03-04 21:00 Copying the File Server D drive is still failing. I shut down the NAS and put a new drive in bay 3.

All the servers are backed up except the File Server D drive which won't copy, but I'm running out of time. So I shut down the NAS and replace the drive in bay 3 with a blank one and the server boot. I rebuild the array offline which is quicker than while it's running but still slow.

2017-03-05 08:00 Array is rebuilt, File Server D drive is still failing

The array is back up and looking healthy. But Veeam is still failing to back up the fileserver D drive.

2017-03-05 19:00 chkdsk /f File Server D overnight

I had to shut all the servers down, unmount the iSCSI drive and run chkdsk /f, it runs over the whole night and finally finishes only to report no errors on NAS. But the Fileserver D drive backups are still failing.

2017-03-06 07:00 chkdsk /R on the NAS

Luckily Monday was a public holiday and I could continue to work leave the VMs shutdown and work on the NAS without causing interruptions to staff, so first thing in the morning I start chkdsk /r. I wrote about Veeam backup errors after NAS hard drive failure.

2017-03-06 20:00 chkdsk finishes. I boot all servers and start a backup.

Finally chkdsk finishes and reports that it's fixed a number of issues. I still don't know for sure that I've fixed the issue with the backups but I'm feeling a little better.

2017-03-07 06:00 File server D finishes the backup.

Finally, I can breathe a huge sigh of relief after a full week of outages and issues everything seems to be running normally again.

Over the next few months

Over the new few months, I worked to decommission this setup and move to a more robust design, including shutting down our VMs and migrating many of our services to the cloud. This system is now thankfully no longer in use and I can sleep easier knowing I will never again need to go through that ordeal.

Also importantly, I've learned the hard way about the importance of proactive maintenance and not having a single point of failure. Fortunately, in this case, there was no data loss, but it came far too close for comfort.

The bicycle side channel

2017-11-30T07:00:00+08:00

There is a problem with most cryptographic systems, it's not new and it affects most systems.

Most formal definitions of a "Perfect security" within cryptography (think One Time Pads) still allow for a few things to be leaked; the size of the messages, the sender and receiver of the messages, and the frequency and number of the messages. These are generally considered out of scope and seen as a protocol problem, not a cryptography one.

When I was first introduced to this problem it was called the "The bicycle side channel". Imagine that Alice and Bob have brought their daughter Eve a bicycle for Christmas along with some other presents. They have wrapped all the gifts up in wrapping paper and placed them Christmas tree.

Eve is curious about what gifts might be coming for Christmas but doesn't want to tear the wrapping paper because then her parents will find out that she has peaked. But she can count the number of presents, and she can look at the labels to see who they are from and who they are for, and with one gift she can tell from the shape that it's a bicycle. Maybe she can't tell what colour it is, or which brand, but even wrapped in paper it still looks like a bike.

Consider the image below.

Author: Brisbane Airport, Photographer Sarah Whyte

Despite the fact that it's entirely gift wrapped, even with a little bow around it. I think most people can still work out that it's an airliner. Those of you who know a lot about aviation might even be able to work out that it's a Fokker F70 based on things like the shape and height of the wings. Sure you might not be able to read the registration number on the tail but you still know what it is.

To bring this metaphor back to cryptosystems imagine the Tor network is just starting off and there are only 5 users currently connected. Four are reading Wikipedia articles, and one is watching YouTube. If you saw a graph of how much traffic was going to and from each node, you could easily work out which user was watching YouTube.

As the size of the network grows this get more complex but there was some serious research into decloaking Tor users with nothing more than Cisco NetFlow. There was another great paper that looked at the information leaked by HTTPS connections just based on the size of the messages.

Some networks such as I2P do take this into account and try to send fixed size (padded) messages at a fixed interval so while the I2P router is running it will be relaying messages or just sending and receiving junk to make it hard to tell when a connection is actively being used let alone who is doing what. Of course, this is a trade-off between performance of the network and secrecy and in system design, you need to choose do you try to foil metadata analysis or do you try for maximum performance.

Installing Duplicati on an Arch Linux Laptop

2017-11-23T07:00:00+08:00

For this setup I'm using Duplicati to backup to Backblaze. In a previous post I've written some instructions on setting up Backblaze.

Installing

To install Duplicati simply run

git clone https://aur.archlinux.org/duplicati-latest.git
cd duplicati-latest
makepkg -si

Then enable and start the service

sudo systemctl enable duplicati.service
sudo systemctl start duplicati.service

Setup Backups

Once duplicati has started browse to http://localhost:8200

The first message I got on Arch was asking if duplicati would be running on a multi-user system. I picked "No, my machine only has a single account".

Now click on Add Backup > Configure a new backup > Next

Give your backups a name, you don't need to use encryption but I'd highly recommend it, and I'd also recommend using their password generator and saving the password in a password manager.

Pick B2 Cloud Storage, put in your Bucket Name, a folder path, your account ID and your application key (from the instructions on setting up Backblaze). It's also a good idea to test your connection before continuing.

Select your source data

Setup a schedule that works for you, the default on of once a day is pretty reasonable for most personal backups.

Unless you have a reason to change them I'd leave the default options.

And your done, you can hit Run now to start the backups or just wait for the schedule to kick in.

It's that easy.

Installing Duplicati on a headless Debian Linux server

2017-11-16T07:00:00+08:00

For this setup I'm using Duplicati to backup to Backblaze. In a previous post I've written some instructions on setting up Backblaze.

SSH into the server

The first thing we have to do is to SSH into the server, because Duplicati will be running as a web service on port 8200 we need to forward connections to that port. Because I've already got Duplicati running on my desktop port 8200 is already taken so I'm going to be forwarding port 8000 on my desktop to port 8200 on the headless Linux server.

In PuTTY open go to Connections > SSH > Tunnels

Set the source port to 8000 and the destination to localhost:8200 then hit Add

Or if your SSHing in from a Linux desktops then use ssh example.com -L 8000:localhost:8200

Installing

Head over to the Duplicati download page and copy the link to the Debian installer (in Firefox you can right click > copy link location)

Then download the file on the server

wget https://updates.duplicati.com/beta/duplicati_2.0.2.1-1_all.deb

Then run the installer using apt (not apt-get)

sudo apt install ./duplicati_2.0.2.1-1_all.deb

If you don't already have Mono installed this will bring a lot of dependencies with it.

Finally enable and start the service

sudo systemctl enable duplicati.service
sudo systemctl start duplicati.service

I found I needed to wait about 45 seconds for the service to start accepting connections.

Setup Backups

Once Duplicati is installed open your web browser and go to http://localhost:8000

Click on Add Backup > Configure a new backup > Next

Give your backups a name, you don't need to use encryption but I'd highly recommend it, and I'd also recommend using their password generator and saving the password in a password manager.

Select your source data

Setup a schedule that works for you, the default on of once a day is pretty reasonable for most personal backups.

Unless you have a reason to change them I'd leave the default options.

And your done, you can hit Run now to start the backups or just wait for the schedule to kick in.

It's that easy.

Installing Duplicati on Windows 10

2017-11-09T07:00:00+08:00

For this setup I'm using Duplicati to backup to Backblaze. In a previous post I've written some instructions on setting up Backblaze.

Install

First head on over to Duplicati and download the latest version for Windows (at the time of this writing that's 2.0.2.1)

Start the installer

Accept the GPL License

Accept the default setup

Click on Install

Hit Finish to launch Duplicati

Launch

Duplicati should automatically launch in your default web browser after installing. If it doesn't, simply open your web browser and go to http://localhost:8200/ngax/index.html

Setup backups

Click on Add Backup > Configure a new backup > Next

Give your backups a name, you don't need to use encryption but I'd highly recommend it, and I'd also recommend using their password generator and saving the password in a password manager.

Select your source data

Setup a schedule that works for you, the default on of once a day is pretty reasonable for most personal backups.

Unless you have a reason to change them I'd leave the default options.

And your done, you can hit Run now to start the backups or just wait for the schedule to kick in.

It's that easy.

Setting up Backblaze for Duplicati

2017-11-02T07:00:00+08:00

I've been looking for new backup software for a few months and I've found Duplicati. I'm absolutely stoked with it, it does everything I want from backup software.

Duplicati can backup to a number of different locations and there are several cloud options but I've chosen to go with Backblaze. I like them because they are extremely open with the design of their Storage Pods. Not just on how they work but releasing all the information you need to go and actually build one yourself.

It doesn't hurt that Backblaze are cheaper than Amazon S3¹ and will mail you a hard drive with your data. Something that CrashPlan used to do until a few years ago.

Signing up to Backblaze

So the first part of setting up Duplicati is setting somewhere to send your backups to, head over to the Backblaze B2 sign up page and click Sign up today

Put in a username and password²

I was told I needed a phone number

So I entered my phone number

and enabled 2-factor authentication

Create a bucket

Next go to Buckets > Create a Bucket

Once the bucket is created click on "Show Account ID and Application Key" then "Create Application Key".

Record this Application Key and keep it safe, we will need it to setup Duplicati

Now we are ready to install Duplicati. Over the next few weeks I'm going to do posts on:

Installing Duplicati on Windows 10
Installing Duplicati on a headless Debian Linux server
Installing Duplicati on an Arch Linux Laptop

and the same price as Amazon Glacier in Sydney ↩
I found that passwords are limited to 50 characters, which isn't great but it's not too restrictive. ↩

Just use subdomains

2017-10-26T07:00:00+08:00

Recently Brian Krebs ran a story about a domain that dell forgot to renew and lost control of for a period of time.

One thing that I noticed from the story was that the domain was DellBackupAndRecoveryCloudStorage.com.

I have no idea why large organisations insist on registering new domains like that, DellBackupAndRecoveryCloudStorage.com could so easily be part of a command and control system just trying to stay stealthy. I remember the recent launch of AmazonLightsail.com, my first though was "is this a phishing domain"? Anyone could have registered these domains, it could be Jo from down the street.

Why not setup BackupAndRecoveryCloudStorage.dell.com or use lightsail.aws.amazon.com? That way it's clear who controls the domain, and makes life easier for network admins that want whitelist, filter or inspect traffic.

Email addresses with apostrophes and mail loops

2017-10-19T07:00:00+08:00

Recently we encountered an issue sending an email from one domain, hosted on Office 365 to another domain also hosted on Office 365. The destination email address had an apostrophe e.g. tim.o'reilly@example.org and we were getting the error message. 554 5.4.14 Hop count exceeded - possible mail loop

When we removed the apostrophe from the email address the email got through ok.

But the error message still didn't make sense to me. Firstly, while rare an email address with an apostrophe is technically valid. And secondly if an address doesn't exist I would expect a 550 5.1.10 Recipient not found or similar message rather than "Hop count exceeded".

As it turned out the apostrophe was a red herring. The destination domain was a hybrid Office 365 deployment, and for any address where the local part did not exist Office 365, the mail was forwarded to the onsite server, which in turn would forward to the Office 365 server.

After getting this error, I tried a few different mail serves with apostrophes and that Office 365, Gmail, EXIM4 on Debian and Yahoo Mail all work ok with apostrophes.

A national identity system

2017-10-12T07:00:00+08:00

A national identity system is something I've been thinking about a lot lately. No real answers here, just musings...

The Problem

There are many departments at all levels of Government that need to track people for various reasons, but each implements its own system and no system is universal;

The status quo

In the United States of America, a Social Security is the de facto standard, CGP Grey made a great video Social Security Cards Explained which covers some of the problems with that.

In Australia,

At the Federal level, there is the Australian Tax Office, with whom most people have a Tax File Number.
At the State level, there are the Licensing Departments, with whom most people have a Drivers License (and specifically a drivers license number)
At the Local level, there are the local councils, to whom homeowners pay council rates and each council has its own way of tracking people.

While things like a TFN, a passport or a drivers license number might cover 90% of the Australian adult population there will be many people that don't have these.

The ultimate ID card

I can imagine a national identity system where people get an ID card that is a veritable Swiss Army knife of modern identity.

The card would have all the usual things on an ID card;

Photo
Full Name
Date of birth
Unique ID number¹
Validity Dates

But the card would also have NFC, with a certificate and private key stored on the card.

The certificate would contain the same information that's visible on the card (e.g. Photo, name, etc...).

Much like with TLS, the certificate would need to be signed by a trusted Certification Authority (e.g. The Federal Government) which would also need to publish a public Certificate Revocation List (CRL) for things like lost or stolen cards.

The utopian vision

To make this vision truly utopian, the standards used by the card would all need to be fully open and public. That way anyone could make use of the cards, not just the Government.

At the moment I've got about a dozen ID cards in my a wallet. An RFID card the building I work in, A Mifare card for our public transport system, my driver's license and so on.

Imagine if I could just give my Unique ID number to my employer, who could add that into the system and I could use my Government issued ID card to open the doors at work. I could use that same card for public transport, my local library, my driver's license², or the Hackerspace down the road.

Better yet if you're using a PC / laptop / smartphone with NFC, the card could act as a universal second factor. You swipe your ID card over an NFC reader, which gives the "Something you have" and automatically populates your username, and then enter your password.

There is no reason this would have to be just one country. If the standards were open and public any nation (or anyone at all) could start issuing compatible cards. Want your system to accept cards from Bangladesh? No problem just add their root CA. Don't trust "Honest Abe's Legitimate Card Issuing Authority"? not a problem, don't add their CA to your root trust.

The possibilities are endless, software companies could build it into their system. What if Adobe Reader (or your PDF viewer of choice) added a way you could use your card to add a cryptographic signature to documents. Just click sign, wave your card over an NFC reader and you're done.

Sure someone could steal your card an sign a document, but it's got to be better than the scanned image of pen on paper that we use now. And that segues nicely into...

Problems with this system

There are many problems with this system, but I feel they fall broadly into two main categories, Privacy implications and Implementation issues.

Implementation issues

I'm not going to dig too much into the implementation issues. Suffice to say implementing a system like this would be a herculean task, wouldn't trust commercial companies with vast resources and great expertise like Google or Apple to implement a system like this without at least some hiccups and flaws. Let alone a federal government agency where this sort of project would instantly become a political football and important bits get outsourced to the lowest bidder.

Privacy implications

More interesting to look at are the privacy implications. All through my utopian vision, I've assumed a benevolent government, one that builds roads, schools, hospitals, provides social services and support for people in need.

But even if we have a benevolent government today, there is no guarantee we won't have a tyrannical dictator next year.

A national identity card would be a very invasive, especially one that could be tracked each time you use it, by making a query back to base. It might not be quite as Orwellian as rolling out a national facial recognition database but in the near future, we are going to have to ask ourselves.

Do we want to trade the privacy that comes with having many, simple, siloed identity systems; for the convenience and efficiency that could come from a unified digital identity system?

Actually I'm thinking that it would need two Unique ID's one on the front that says the same for the life of the cardholder, and one on the back that's unique for each card, and would be the fingerprint of the public key. ↩
The idea here is you could read the NFC tag on a tablet which queries a license database (over a public API) and a screen pops up with my name, photo and what types of vehicles I'm allowed to drive. To stop people bulk querying the database, each query would need to be signed by the private key on the card it's looking for, so you would need physical access to the card to query the details. ↩

SEO spear spamming

2017-10-05T07:00:00+08:00

I've recently started receiving a new type of spam¹. It's fairly targeted, so like spear phishing, I'd call this stuff spear spamming.

Instead of regular spam which usually falls into categories like "Buy pills online", "Meet single ladies tonight" and "Open this malware laced attachment" this one is sent by someone trying to improve their search engine rankings or SEO. The emails follow a fairly standard formula with a few deviations.

Hi {Name}

I just finished going through your article here: {url of a post I've written} Thanks for the resource!

I noticed you mentioned {url of a competitors product, that I've linked to}.

I've recently written up a comprehensive and up-to-date 3,000 word review of {general subject area} that I think your readers would be very interested in.

Check out the post here {url of post looking for SEO}.

Would you consider linking to it in the article of yours I mentioned above? I saw you liked to {competitors product} so I figured I'd see if you'd link to mine as well. Perhaps your visitors would find it helpful.

Kind Regards,

{Name of Author of post looking for SEO}

P.S. I respect the relationship you have with your readers, I wouldn't ask you to link to anything I didn't think was an excellent resource for you guys.

This is then followed up by a second email exactly² a week later;

Hello again -

I figured I'd try one more time :)

Did you happen to get my last email? I imagine you are super busy and ...

I received one of these messages for a post I'd written about Tunneling data over DNS, in the footnotes of that post I give credit saying "This network diagram was drawn with draw.io" and I got an SEO email saying they were sure my readers would love to learn more about drawing and art.

Google has been pretty open about the fact that the best way to increase a sites page rank is to get other websites to link to it. So it makes sense that people would scrape the web looking for sites that link to similar content and ask for a link to their site.

The emails look very good but there are a few telltale signs that there automated. One of the emails I got had an unsubscribe link at the bottom. And all of the emails I've seen have been sent using Google's Gmail API and have the header

Received: from {random id} named unknown by gmailapi.google.com with HTTPREST;

By that, I mean new to me, this stuff has probably been around for years but I have only just started seeing it. ↩
In my experience, it's always been within one hour of exactly 7 days. ↩

Code reuse is good for security

2017-09-28T07:00:00+08:00

I was listen to a podcast¹ recently and Stephen Ridley from Senrio said "code reuse is vulnerability reuse" and I don't like that.

I don't think Stephen is wrong, I think he is correct in a way and he is a very smart person. The problem is that you just can't squeeze a huge amount of subtlety and nuance into a three second soundbite.

I believe we are better off overall when we do reuse code. For example I was recently working on a project where I had to take untrusted user input formatted as CommonMark and render it as HTML. I could have tried to write my own parser to do that but instead I used the CommonMark-py library to change the CommonMark to HTML and then used Mozilla bleach to whitelist only the HTML tags we want.

Either of those libraries could have horror show bugs in them just waiting to be found and get me pwned, but I'd trust Mozilla to do a better job of sanitizing HTML than I can.

Or in the specific example they were talking about on the podcast; A number of IoT devices were using the gSOAP library and there was a remote code execution bug in the library which affected millions of devices. That's bad. But I still think these devices were better off using an existing library than each different vendor trying to write their own SOAP library and getting it wrong in their own unique way².

Importantly though you need a way to track and patch all your libraries quickly and painlessly when issues do inevitably come up.

While code reuse is vulnerability reuse, I'd rather be responsible for software with a large number of (well supported, actively tracked and easily patched) 3rd party libraries than need to look after software where everything is written in house.

On balance I think that "code reuse is good for security"

Risky Business #465 At 43:28 in. ↩
Although this brings up another issue which is that monoculture is bad, because when a vulnerability does crop up it can spread like wild fire. ↩

Looking through GitHub's DMCA takedowns

2017-09-21T07:00:00+08:00

GitHub publish a copy of all the DMCA takedown notices GitHub receives.

One thing I found very interesting was looking at their graphs showing the number of commits over time.

Of course, this doesn't necessarily mean that DMCA takedowns are being issued more often in general, it could simply be a reflection of GitHub's own growth over time. Personally, I suspect it's a mix of the two, both that GitHub is becoming a more popular place to store things online, and that more and more DMCA takedown notices are being issued every day.

I think that our copyright system is badly broken, in particular the approach that many sites (not specifically GitHub¹) are forced to take is very "guilty until proven innocent" where content is removed first with the option to appeal later, and there is little recourse against organisations that submit inaccurate takedown requests.

However, I find it hard to get too worked up about the notices on GitHub, having looked through a few of them, most seem to be fairly tightly targeted and aimed at a single repository of clearly infringing content. In fact I was surprised how many of them were for .pdf copies of books and not for code at all.

GitHub actually have the phrase "Users identified in the notices are presumed innocent until proven guilty", although I'm not sure exactly how that works. If they are getting less than 40 per day it's conceivable they have a real human being looking at each of the notices. Actioning the ones that are simple and investigating or flagging ones that might be a grey area. ↩

Does Microsoft's Office 365 licensing model encourage poor security practices?

2017-09-14T07:00:00+08:00

I've recently survived a migration to Office 365 with relatively few scars and coming out only slightly more cynical, bitter and twisted than I was when I when I started¹.

I remember speaking to a Microsoft sales rep at some conference or other, they said that in Office 365 you only need to buy one license per users. That's it, and a user was unofficially defined as a living breathing bag of meat that has an Office 365 account.

No more worrying about buying CALs for printers and photocopiers
No need to buy an extra license for users who had a regular account and an administrative account
No extra CALs because someone wants their emails on their phone and their laptop at the same time
No need to license accounts used by system scripts and scheduled tasks.
No worries about Per-Core vs Per-CPU licensing
No need to license unused CPU cores because a VM could potentially be migrated in a failover situation.
No weird definition of "User"²

Just one license per person. Simple. I was quite surprised.

It turns out that's not quite correct, while you don't have to license MFDs. In practice, if you want your photocopiers and scanners to be able to authenticate and send email using Office 365 they are going to need an account.

There is a work around for this, I could setup a shared account scanner@example.com, shared accounts don't need a license. I could give my account Michael.VanDelft@example.com 'Send As' permission. Then I can then setup the photocopiers to authenticate with my credentials and send email as scanner@example.com. While that's totally ok from an Office 365 licensing perspective I'm left with my username and password stored in a bunch of poorly secured photocopiers and I can't change my password without breaking the scan to email function on all our photocopiers.

There are some other work arrounds for this, but they either involve direct send or running an SMTP relay and neither option is great.

We have a plethora of things which send email, not just photocopiers but system monitoring tools, our backup software sends reports, several system scripts, our financial system, even our firewall emails alerts occasionally.

If we want these all these things to send emails through Office 365 so we get DMARC, TLS, Authentication and all that other goodness that comes with a well-configured mail server we need to license them.

Although it's possible that I've almost reached peak cynical saturation and simply couldn't have got any more bitter and twisted even if it had been a migration to Oracle's god-awful "Oracle Communications Messaging Server". ↩
We were once looking at Microsoft SQL server and we were told that if we had some software (e.g. accounting software) that used a single account to connect to the database, anyone who used that software was a "user" of the database, even if we only had one account setup in SQL server. Fair enough, I guess. But then they extrapolated that to say that if we used a CMS like WordPress with Microsoft SQL server as the back end, then we would need a license for everyone who viewed or commented on our website as they too would be a "user" of our database. Needless to say, we went with Per-Core licensing rather than per user. I've since been told that this is not correct by a number of "Microsoft Licensing Experts" however it's what we were told at the time by our reseller who was a "Microsoft Gold Certified Partner". ↩

CrashPlan personal is shutting down

2017-09-07T07:00:00+08:00

I’m currently using CrashPlan to back up my home server, desktop, laptop and a few computers for family members. I’ve been fairly happy with it and it ticks most of the boxes. But I recently got the email saying that CrashPlan for home will be shutdown.

So now I'm on the hunt for some new backup software. The features I’m looking for are:

Must be Cross Platform (Window and Linux, I don’t need Mac but it would be nice)
Must be able to run on a headless Linux server
Must be able to do incremental backups at the file level (e.g. If I’ve got a 20GB QCOW2 file and 100Mb of that changes I don’t want to reupload 20GB)
Must be able to restore file revisions (not just the latest version like if I just used Rsync)
Must be able to encrypt backups locally before uploading them
Should be able to upload to some sort of “Cloud” offering. I don’t mind whether that’s the backup vendor’s infrastructure or something public like Amazon S3
Should be able to do continuous backups (CrashPlan could run every 15 minutes, that's pretty good)
Would be nice if it was also able to back up to a server I control or other desktops.
Would be nice if it could backup to a physical drive (external USB hard drive)
Would be nice if it was open source
Would be nice if it also had a pretty GUI, although I’d take an ugly and difficult system that's set and forget over a pretty GUI that needs tending regularly.

I realise that’s quite a long list but other than headless Linux support I think most of it is pretty mainstream requirements for a backup tool.

Are outbound firewalls worthwhile?

2017-08-31T07:00:00+08:00

I was recently setting up a server on Microsoft's Azure platform from work and by default Azure pick I high port number¹ and NAT it to port 3389 for RDP. I must have spent a good hour trying to work out why I couldn't connect to the server when I finally realized that I hadn't unblocked the port I was trying to connect on in our outbound firewall.

Now I'm skeptical of the value of changing port numbers to hide services anyway but also don't know how much value, if any our outbound firewall add either.

On the one hand, we allow ports 22 and 443 outbound so it's easy for someone to tunnel over ssh or proxy through a HTTPS website. Basically, if someone wants to connect out of our network, the firewall not going to stop them.

On the other hand, we don't open port 23 so if someone brought in a home route infected with Mirai and it tried to telnet out to infect others, or some other dumb worm that uses a port we don't have open then our firewall would at least block that.

So like many defenses in depth things, by itself, it's not going to make much difference, but it's one more layer that malware needs to get around.

I read somewhere that it's a random number in the range 49152-65535 but couldn't find official documentation to back that up. ↩

Review of Fatal Flight: The True Story of Britain's Last Great Airship

2017-08-24T07:00:00+08:00

I've just finished listening to Fatal Flight: The True Story of Britain's Last Great Airship and it is, in a word "Brilliant".

When I started this blog I intended to put up several book reviews, a year and half later, this is my second book review.

Bill Hammack is an incredibly eloquent author and can turn the most seemingly mundane of topics into a fascinating story.

In the book, Bill explores the story of R101 "Britain's Last Great Airship" the people that built it and why it ultimately failed¹.

As an engineer, Bill is well equipped to look at the technical details, but he also does an outstanding job of examining the political reasons why airships failed, both for R101 specifically and British airships generally.

While the book is not related to information security, in fact, it's not even related to information technology, it's one of those books that I think would enrich the knowledge of anyone working on large projects be they engineering or otherwise.

Bill has generously released the audiobook as Creative Commons and I would encourage anyone to go over to his site and download a copy.

This was partly the inspiration for last weeks post on learning from failure ↩

Learning from failure

2017-08-17T07:00:00+08:00

Recently I watched a video by Tom Scott about a Museum of Failure. It's a fascinating video and well worth watching.

Often there is a lot of emphases put on learning from the great successes, seeing what successful people or projects have done and trying to emulate them. But learning from failure is just as important and often overlooked.

Seeing where projects have gone wrong and how to avoid those weaknesses in the future can be incredibly beneficial.

I remember being told once that

You never really understand the value in things like project management and it's easy to see them as needless overhead, until you have worked on a project that has failed.

Reuse before buy before build

2017-08-10T07:00:00+08:00

Yesterday I was at a CIO forum and one of the presentations was about Enterprise Architecture Planning within the West Australian State Government. They were pushing the idea that government departments should work together as a single organisation rather than as separate and sometimes competing entities and they used the phrase:

Reuse before buy before build.

I think that's great.

If there is existing software that you or another department has developed or have already licensed, or there is open source software available use that first.

Only if reuse is not possible, then look at buying an off the shelf solution.

Finally, if there is no existing open source solution and no off the shelf solution, then develop a new solution.

Installing software from the Arch User Repository

2017-08-03T07:00:00+08:00

This is another note to self, it is very simple and I've done this a few times before but every single time I need to go back to the documentation.

In this example I'll install heimdall which is used for flashing new firmware onto samsung phones.

First go to https://aur.archlinux.org/ and type the name of the package into the package search box.

This takes us to https://aur.archlinux.org/packages/heimdall-git then copy the git clone url from the top. In my case I've got a folder setup for AUR installs because I like things to be organised but it's not necessary.

clone the repository, cd into it, then run makepkg -si. makepkg should not be run as root, when root permissions are needed you will be prompted for your password if you have sudo or the root password otherwise.

git clone https://aur.archlinux.org/heimdall-git.git
cd heimdall-git
makepkg -si

and that's it. It really is very simple.

Then later to remove the package and it's dependancies you can run

sudo pacman -Rs heimdall-git

You can also run pacman -Qem to get a list of manually installed packages which for most people will be just the packages they have installed from AUR.

Veeam backup errors after NAS hard drive failure

2017-07-27T07:00:00+08:00

We have a setup at work where we have two Hyper-V servers in a failover cluster mounting a VHD on a NAS¹ as an iSCSI target running a number of VMs that get backed up by Veeam.

Recently we had a hard drive failure in the NAS. The NAS had RAID 5 with a hot spare² so no problem, in this case, the drive died on a Friday so I shut the server down on the weekend, popped a new drive in and rebuilt the array offline.

All looked good, the RAID controller said all drives were OK and the servers started back up fine, but the next night the Veeam backups failed with the error message.

2017-07-23 9:16:59 PM :: Processing HWFS1 Error: Incorrect function.
Failed to read data from the file [\\?\GLOBALROOT\Device\CSV{8144a28c-459c-41a7-a274-b03ae6a3d493}\HWFS1_D.vhd].
Failed to read data from the file [\\?\GLOBALROOT\Device\CSV{8144a28c-459c-41a7-a274-b03ae6a3d493}\HWFS1_D.vhd].
Failed to upload disk.
Agent failed to process method {DataTransfer.SyncDisk}.
Exception from server: Incorrect function.
Failed to read data from the file [\\?\GLOBALROOT\Device\CSV{8144a28c-459c-41a7-a274-b03ae6a3d493}\HWFS1_D.vhd].

After some digging, I found events in the event view on the NAS that said.

Logical drive 2 configured on array controller P212 located in server slot 3 returned a fatal error during a read/write request from/to the volume.  

Logical block address 4123043672, block count 8 and command 32 were taken from the failed logical I/O request.  

Array controller P212 located in server slot 3 is also reporting that the last physical drive to report a fatal error condition (associated with this logical request), is located on bus 0 and ID 11.

So I ran chkdsk /f e: on the server and it said

Windows has checked the file system and found no problems.

But I was still seeing the error messages in event viewer. Then I tried chkdsk /r e: which took several hours but eventually came back and reported that it had found and repaired (moved) several unreadable sectors.

So even when every thing seems ok, and chkdsk reports no errors if you get a Failed to read data from the file message from Veeam it could be bad sectors on the underlying disk.

This is not a good design for a number of reasons, we are migrating away from it, but that's a blog post for another time. ↩
Also not great design. ↩

A morbidly fascinating look at Australian causes of death

2017-07-20T07:00:00+08:00

Recently a discussion came up on the ITPA discourse, where the Australian Federal Government is trying to get backdoor¹ access to the content of encrypted messaging apps.

This included a great quote² from our esteemed leader Malcolm Turnbull:

"The laws of mathematics are very commendable but the only laws that apply in Australia is the law of Australia."

Patrick Gray has a great write-up where he suggests that what the Government are really after is a way to compel companies like Apple and Google to push a rogue update to targeted handsets that will allow law enforcement access to the decrypted communications on the device itself.

Inevitably the question of whether all the recent anti-terror legislation passed by the Australian Federal Government was saving lives, and how many people had been killed by terrorism with one member stating more people had been killed by knives than by terrorism.

yet we don't see the government trying to ban knives, or legislating that only blunt knives can be sold.

While I completely agree with the above, I thought it would be interesting to look through some hard data³.

I went to the Australian Bureau Statistics website where they publish ten years worth of cause of death data in Australia. I downloaded the Underlying cause of death, All causes, Australia spread sheet and started reading through.

Unsurprisingly ABS doesn't list "terrorism" as a cause of death, I suspect any terrorism related deaths would be under "CHAPTER XX External causes of morbidity and mortality (V01-Y98)", then under "Other external causes of mortality (X60-Y36)", then "Assault (X85-Y09)" and finally under whichever specific heading was relevant. For example "Assault by explosive material (X96)" or "Assault by rifle, shotgun and larger firearm discharge (X94)".

Because I was unable to work out which deaths were from terrorism in the ABS statistics, I then found a fairly well sourced Wikipedia article from which we get the following table

Year	Number of incidents⁴	Deaths	Injuries
2006	2	0	0
2007	0	0	0
2008	3	0	0
2009	1	0	0
2010	1	0	0
2011	0	0	0
2012	0	0	0
2013	1	0	0
2014	8	4	7
2015	7	2	0
Total	23	6	7

Which gives us 6 deaths over the same 10 year period as the ABS statistics. Of the 1,454,112 deaths in that period that's about 0.0004% were from terrorism, it slips in just behind "Other disorders of penis (N48)" killing 7 people and "Inflammatory disorders of male genital organs, not elsewhere classified (N49)" taking the lives of 64 men between 2006 and 2015.

Terrorism absolutely pales in comparison to our top category⁵ "CHAPTER IX Diseases of the circulatory system (I00-I99)" with a whopping 31.4% totaling 456,956 deaths over ten years.

Not all, but many deaths from diseases of the circulatory system are preventable. Tens of thousands of lives, maybe even hundreds of thousands of lives over that same ten year period could have been saved with greater investment in Health Promotion⁶ and tougher laws⁷ on tobacco, alcohol and fastfood.

They say they "don't want a backdoor", what they want is something different, and then proceed to describe a system that by any reasonable definition is a backdoor. ↩
In fairness to Malcolm, if you followed me around with a camera all day and made me do several press conferences on a number of different subjects, you wouldn't have to wait long for me to say something gobsmackingly stupid too. ↩
I fully admit that I went into this with a theory and was simply looking for data to back up what I already believed. So while this data is accurate, I didn't put much effort into looking for data which refuted my world view, you can take this with a grain of salt if you like. ↩
Of course there is the question of when does something change from simply crime to terrorism? When does a boat become a ship? How long is a piece of string? but I'm happy to go with whats in the Wikipedia article. ↩
I know that "Diseases of the circulatory system" is a whole broad category of causes rather than a single cause, but I would say that "Terrorism" is a whole category too, it's not broken down into individual causes so it's a fair comparison. ↩
Full disclosure, I work for The West Australian Health Promotion Foundation. Also in case any readers lack common sense, My views that I express on my personal blog are my own, and not those of my employer. ↩
To be clear here, when I say "tougher laws" I don't support prohibition. If informed adults want to put something into their own body that's up to them. I'm talking about things like plain packaging, health warnings, advertising restrictions and higher taxes. The devil is in the details but I support the idea of a sugar tax for example. ↩

Crashplan no longer offer restore to door service

2017-07-13T07:00:00+08:00

A while ago I got an email from someone who was Googling for rsync crashplan and stumbled upon a previous blog post about backing up servers with rsync then using Crashplan to backup those files.

They were desperately looking for some kind of rsync-like solution to restore a lot of data from Crashplan over a bad connection.

My initial thoughts were that Crashplan had a service where they would send you your backups on a hard disk. The old "never underestimate the bandwidth of a station wagon full of tapes" sort of thing.

But after a bit of searching, I found that their Restore-to-Door service was discontinued in early 2016. I was frustrated that I was unable to help the person who had contacted me, but also a little worried that I might end up in the exact same situation.

At the moment I've got a Family license for Crahsplan which for $165 AUD a year covers up to 10 computers. I've got it installed on my several of my computers as well as my mums, uncles, and fiancée. Some of those computers have over 300GB of stuff on them and to download all of that again over the dodgy ADSL we get in Perth would take the better part of a month.

I'm not sure what I'm going to do, but I am seriously considering finding somewhere locally to host my own Crashplan server, that I can get physical access to and not paying for Crashplan's cloud offerings anymore.

Sleazy marketing

2017-07-06T07:00:00+08:00

I was recently looking at a number of application whitelisting solutions and one of the ones I was looking at was Carbon Black. I spent quite a bit of time on their website trying to see if they actually published any solid, useful, technical documentation about what it is they actually do beyond their tagline "Stop the Most Attacks. See Every Threat. Compromise Nothing."

A few days later I got a call from a sales person from Carbon Black saying that they saw I was interested in their product.

I assume they have some algorithm on their website analytics that does a lookup on any IP address that spends more than a set amount of time on their site (I was there for about 10 minutes). If you do a PTR lookup on our gateway IP address or throw it into any Geo IP database like ipinfo our organisation comes up. From there it's not hard to Google us, call reception and ask for whoever manages IT Security.

I browse the web with Do Not Track switched on, I understand that it's voluntary and websites can just ignore that flag. But tracking me shows a complete lack of respect for users privacy wishes and doesn't inspire me to install their products on all of the desktops I manage.

Bruce Schneier blogged about something similar recently where websites were grabbing user form data before it's submitted. He says

"This is important because it goes against what people expect"

Just like using javascript to grab from data, tracking users is not that technically difficult but it's not what people expect. Browsing someone's website is not the same as filling in the contact us form and you don't expect to get a call from one of their marketing people.

Setting Up Full Disk Encryption on Debian 9 Stretch

2017-06-29T07:00:00+08:00

Previously I did a tutorial on Installing Debian 8 Jessie with full disk encryption, in that tutorial I went into a lot of detail about manually partitioning the disks. If for some reason you want to manually partition your disks I would reccomend that tutorial, it will still work for Debian 9 Stretch.

However this tutorial is much more simplified I've used the grapical installer and gone with "Guided - use entire disk and set up encrypted LVM".

As I've mentioned in all my tutorials on Full Disk Encryption I say "Full" disk encryption but that's not entirely correct there is still a small partition /boot that's unencrypted. That contains your kernel, grub config and initrd and needs to be unencrypted so we can start booting and decrypt the rest of the OS.

So let's get started

Installing

Boot up your CD, USB flash drive, ISO file or install media of choice and select Graphical install.

Select your language.

Select your location.

Set your keyboard layout.

Pick a name for your computer.

Set your local domain.

I personally leave the root password blank, this disables the root account and instead sets you up with a first user that can run sudo to become root.

Enter your full name.

Pick your username (the default is usually pretty good).

Set your password.

Set your timezone.

The encryption

This is where the magic happens, actually it's quite simple, we are going to pick "Guided - use entire disk and set up encrypted LVM" and then just go with the defaults. As I said before if for some reason you want to manually partition your disks I would reccomend a previous tutorial.

Select the volume to install Debian. (This will wipe whatever you have on that disk!!)

Pick "All files in one partition (recommended for new users)".

Pick 'Yes' to write the changes to the disks.

Now the disk will be writen with random data, this is to prevent analysis of the disk. This step can be skipped by pressing cancle but it's highly reccomend you wait it out. It could take several minutes to a few hours so now is an absolutely smashing time to go and have a cup of tea.

Now set a passphrase for your disk.

Select "Finish partitioning and write changes to disk"

Pick 'Yes' to write the changes to the disks.

Continue the installation

Now we continue the installation as per normal.

Pick 'No' for any extra CDs.

Pick your country to find a local mirror .

And pick your mirror of choice, often (at least in Australia) you will find your local ISP has a mirror and this will likely be fastest for you.

Enter any proxy information (most times this will be blank)

You are given the option to opt-in to Debian's statistics collection.

Pick your software, I've gone with KDE as my desktop of choice but it's a matter of personal taste.

Install GRUB

Pick your boot disk.

and finish the installation.

Boot your system

Now when you boot up you should presented with a prompt asking for the key to decrypt sda5_crypt (your encrypted volumne)

Enter your passphrase (Note: you won't see characters as you type)

Now you can log in and enjoy your new Debian system

Installing OpenCanary on a Raspberry Pi

2017-06-22T07:00:00+08:00

A few recent Risky Business podcasts have been sponsored by Thinkst and they have been plugging their Canary tools. Basically, little honeypots that sit on your network and sends an alert when something tries to access them. To me, the idea sounded pretty cool but when I looked at their pricing it said

For under $10k, you get 5 Canaries, a dedicated console, and 5 licences for alerts, support and maintenance.

While for some organisations $10,000¹ might be chicken feed for me that's prohibitively expensive. So I looked around for open source alternatives and was surprised to find that Thinkst have released OpenCanary. It doesn't seem to be getting a whole lot of love with only a few commits in over a year at the time of this writing but I did have a spare Raspberry Pi and it's open source so if something is missing I can make a pull request.

Grab a Raspberry Pi and Install Raspbian

Download the Raspbian Jessie Lite image and SSH in. There are already hundreds of tutorials so I'm going to skip this step and just assume you have a fresh Raspbian install that you can SSH into.

If you haven't already, update all your packages.

sudo apt-get update && sudo apt-get dist-upgrade

Install the prerequisites

Install the packages needed to build OpenCanary.

sudo apt-get install git python-virtualenv python-pip python-dev libssl-dev libffi-dev

Install a virtual environment

It's recommended that you run OpenCanary in a virtual environment. It makes managing libraries easier but if the only thing your going to run on the Raspberry Pi is OpenCanary it's not strictly necessary.

virtualenv -p python2 canary-env
source ./canary-env/bin/activate

The versions of pip and setuptools that come with Debian's virtualenv are a little out dated and need to be upgraded for OpenCanary

pip install --upgrade pip setuptools

Clone the git repository

git clone https://github.com/thinkst/opencanary
cd opencanary

Install OpenCanary

python setup.py install

I got some build errors with Jinja2 but it's a known issue and does not impact OpenCanary.

Also building cryptography and the other dependencies took about 10 minutes on my Raspberry Pi so now is an absolutely smashing time to go and have a cup of tea.

Setup config and start OpenCanary

OpenCanary does have a --copyconfig option which creates a config file in your home directory, however, I found that sometimes OpenCanary misses the config file in the home directory. I tried debugging it but in the end found it more reliable (and logical) to save the config to /etc/opencanaryd/opencanary.conf

sudo mkdir /etc/opencanaryd
sudo cp opencanary/data/settings.json /etc/opencanaryd/opencanary.conf

For some reason when I installed OpenCanary the opencanary.tac file did not copy across correctly and I kept getting an error

Unhandled Error
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/Twisted-14.0.2-py2.7-linux-armv7l.egg/twisted/application/app.py", line 642, in run
    runApp(config)
  File "/usr/local/lib/python2.7/dist-packages/Twisted-14.0.2-py2.7-linux-armv7l.egg/twisted/scripts/twistd.py", line 23, in runApp
    _SomeApplicationRunner(config).run()
  File "/usr/local/lib/python2.7/dist-packages/Twisted-14.0.2-py2.7-linux-armv7l.egg/twisted/application/app.py", line 376, in run
    self.application = self.createOrGetApplication()
  File "/usr/local/lib/python2.7/dist-packages/Twisted-14.0.2-py2.7-linux-armv7l.egg/twisted/application/app.py", line 441, in createOrGetApplication
    application = getApplication(self.config, passphrase)
--- <exception caught here> ---
  File "/usr/local/lib/python2.7/dist-packages/Twisted-14.0.2-py2.7-linux-armv7l.egg/twisted/application/app.py", line 452, in getApplication
    application = service.loadApplication(filename, style, passphrase)
  File "/usr/local/lib/python2.7/dist-packages/Twisted-14.0.2-py2.7-linux-armv7l.egg/twisted/application/service.py", line 405, in loadApplication
    application = sob.loadValueFromFile(filename, 'application', passphrase)
  File "/usr/local/lib/python2.7/dist-packages/Twisted-14.0.2-py2.7-linux-armv7l.egg/twisted/persisted/sob.py", line 210, in loadValueFromFile
    exec fileObj in d, d
  File "/usr/local/bin/opencanary.tac", line 4, in <module>
    __import__('pkg_resources').run_script('opencanary==0.3.2', 'opencanary.tac')
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 531, in run_script
    name = ns['__name__']
exceptions.KeyError: '__name__'

Failed to load application: '__name__'

I needed to copy the tac file manually.

cp bin/opencanary.tac /home/pi/canary-env/bin/opencanary.tac

I also found the default Raspbian image has the NTP service running and so port 123 was already in use. I chose to disable the NTP module in OpenCanary

sudo nano /etc/opencanaryd/opencanary.conf

"ntp.enabled": false,

Alternatively, you could leave the NTP module enabled and disable the service on the Raspberry Pi

sudo systemctl stop ntp.service
sudo systemctl disable ntp.service

Start OpenCanaryd

As a bit of a "Hello World!" start opencanaryd in developer mode so it runs process in the foreground to check it's all working

opencanaryd --dev

Hopefully, you will see a message that contains Canary running!!! although you will probably also see a number of Dropping log message due to too many failed sends messages as well. This is because opencanaryd trying to send messages to opencanary-correlator but we don't have that setup yet.

At this point you can have a play with your canary, try to nmap it or telet to it and see the output.

Once you have had some fun Ctrl + C out to close opencanaryd

Setup Email Alerts

Apparently, it's possible to have your canary log directly to email but when I tried I couldn't get it to work.

I looked at opencanary-correlator, but it uses mandrill for mail and that's now a paid MailChimp add-on which I didn't want to use.

In the end, I found it quicker and easier to write a simple python script to work like correlator and forward all alerts to an email address.

sudo nano canary_log_forwarder.py

Add your email addresses and SMTP server into the script and save it.

"""
Forwards logs from OpenCanary that come in on port 1514 to an email address.

This is a very simple script, it does no validation on the logs, it just
forwards everything that comes in.
"""
import smtplib
from email.mime.text import MIMEText
from email.mime.multipart import MIMEMultipart

from twisted.internet import protocol, reactor

# Settings
FROM_ADDRESS = 'opencanary@example.com'
TO_ADDRESS = 'security@example.com'
SMTP_SERVER = 'mail.example.com'
# Saving passwords in a file is not a great idea. If you do need to log in to
# your SMTP server, at the very least make sure this file is not world readable
# e.g. `chmod 700 canary_log_forwarder.py`
SMTP_USERNAME = None
SMTP_PASSWORD = None


class SendEmail(protocol.Protocol):
    def dataReceived(self, data):
        message = MIMEMultipart('alternative')
        message_body = MIMEText(data, "plain", "utf-8")
        message['Subject'] = 'Alert from OpenCanary'
        message['From'] = FROM_ADDRESS
        message['To'] = TO_ADDRESS
        message.attach(message_body)

        server = smtplib.SMTP(SMTP_SERVER)
        server.ehlo()
        server.starttls()
        server.ehlo()
        # Login if applicable
        if SMTP_PASSWORD and SMTP_PASSWORD:
            server.login(SMTP_USERNAME, SMTP_PASSWORD)
        server.sendmail(FROM_ADDRESS, [TO_ADDRESS], message.as_string())
        server.quit()


class EmailFactory(protocol.Factory):
    def buildProtocol(self, addr):
        return SendEmail()


reactor.listenTCP(1514, EmailFactory(), interface='localhost')
reactor.run()

Make it a service

Now we have everything setup we want to make it run as a service and start automatically when we boot up the Raspberry Pi. So we will create two systemd .service files.

sudo nano /etc/systemd/system/opencanary.service

[Unit]
Description=OpenCanary honeypot
After=syslog.target
After=network.target

[Service]
User=pi
Restart=always
Environment=VIRTUAL_ENV=/home/pi/canary-env/
Environment=PATH=$VIRTUAL_ENV/bin:/usr/bin:$PATH
WorkingDirectory=/home/pi/canary-env/bin
ExecStart=/home/pi/canary-env/bin/opencanaryd --dev

[Install]
WantedBy=multi-user.target

sudo nano /etc/systemd/system/canary-log-forwarder.service

[Unit]
Description=Canary log forwarder
After=syslog.target
After=network.target

[Service]
User=pi
Restart=always
Environment=VIRTUAL_ENV=/home/pi/canary-env/
Environment=PATH=$VIRTUAL_ENV/bin:$PATH
WorkingDirectory=/home/pi/canary-env/
ExecStart=/home/pi/canary-env/bin/python /home/pi/opencanary/canary_log_forwarder.py

[Install]
WantedBy=multi-user.target

sudo systemctl enable canary-log-forwarder.service opencanary.service
sudo systemctl start canary-log-forwarder.service opencanary.service

Finshed

Your canary should now be all set up and ready to run.

It's a good idea to reboot it just to make sure all the services start correctly.

sudo reboot

You should get emailed when it boots up letting you know that all the services have started.

I know the website said "Under $10k" but they wouldn't phrase it like that if the price was $300. ↩

Why I like the MIT License

2017-06-15T07:00:00+08:00

Recently I've spent a little too much time thinking about open source licenses.

Whenever I start a new project I struggled to pick a license, there are dosens open source licenses that are all more or less the same. In fact worese than being the same many licenses that have the same goals are subtlety incompatible with one another.

Linus has expressed why he likes GPLv2 and not GPLv3. Even within BSD license there are diffrent versions which can be confusing.

The MIT license however is, short, simple, open, doesn't have lot's of diffrent variants and I was supprised to find it's the most popular license on GitHub so it's very well known.

ASD's Essential Eight

2017-06-08T07:00:00+08:00

I've long been a fan of the advice from the Australian Signals Directorate (ASD) [previously the Defence Signals Directorate (DSD)]. Not too long ago they changed their "Top 4" to their "Essential Eight".

What I like about ASD's advice is that it's easy to read, in comparison ISO 27001:2013 might be full of great advice but even the name is indecipherable jargon to most people.

ASD's Essential Eight are simple to understand, and with the exception of Application whitelisting, they are relatively easy to implement. They are:

Application whitelisting
Patch applications
Disable untrusted Microsoft Office macros
User application hardening (Uninstall shovelware)
Restrict administrative privileges
Patch operating systems
Multi-factor authentication
Daily backup of important data

While it might be fun to install blinky-light boxes that run fancy machine learning algorithms and cost a fortune. ASD's Essential Eight are cheap, simple and effective and will definitely get you your best bang for your buck.

Designing for Failure

2017-06-01T07:00:00+08:00

I recently purchased my grandfather's house which he built himself in the mid 1950s. My fiancé and I were painting the house and we came across a pipe sticking out of the wall just above the rain water tank.

Looking inside the roof we discoved that this pipe came from a large metal tray that was sitting underneath the hot water system. The manufacturers of the hot water system never intended for the tank to leak¹ but it's something my grand father clearly though about and designed for just incase.

Simiarly on the roof there is a copper pipe that looks a bit like a shepherd's crook that comes out of the hot water system.

The idea is that if the tank somehow got overfilled and then the water got boiled and expanded, rather than the tank exploding with the pressure, the boiling water would be forced up the pipe and pour out on the roof. As far as I'm aware the tank has never been overfilled or over boiled, but it's good to have a contingency just incase.

In IT we are getting better at designing for failure, things like RAID or small office routers that now come with both DSL and a 4G modem so if the DSL dropps out it switches over.

In Information Security we have ideas like defence in depth and layered security. They are a good start but we need to keep working on them.

And to their credit the tank is older than I am and still going well. ↩

Fair use and copyright reform in Australia

2017-05-25T07:00:00+08:00

I've just finished listening to the audio book of Cory Doctorow's Information Doesn't Want to Be Free. It's a fascinating read and I'd recommend it to anyone interested in copyright.

Admittedly some bits are a little repetitive but it really hammers home how broken the current copyright system is and how much damage archaic copyright laws are doing to the internet.

At the same time Electronic Frontiers Australia, a number of Australian Wikipedians and the Australian Digital Alliance have released a new website faircopyright.org.au which is campaigning for more sensible copyright laws in Australia.

I personally would be willing to buy a copy of Information Doesn't Want to Be Free for any Australian MP who is willing to listen to or read it and I'm planing on sending an email to my local MP to that effect.

If you're not paying for it you're not the customer you're the product

2017-05-18T07:00:00+08:00

There are exceptions to every rule. I'm certainly not saying every free service exists to profit from it's users. I believe in charity and people doing things for the common good. Wikipedia and the Linux Kernel are great examples.

But even with Wikipedia, I would say the "customer" are people who donate¹ and what they are "buying" is a world where everyone² has access to information.

That might be pushing the metaphor a bit too far, but I think it's always worth thinking about who is paying the bills of any free service.

Even this blog, I'm paying the bills³, and the customer isn't the people reading the blog it's me, I'm paying for a platform where I can ~~rant~~ express my self.

As a disclaimer, I have donated to Wikipedia a few times when they do their funding drives. I think it's a worthy cause and I would encorage others to do so too. ↩
Everyone with an internet connection. For now, but we buy into the dream that one day everyone in the world will have access to information. ↩
In case you're wondering, I pay $20 AUD a month on hosting with RansomIT and $70 USD per year for the domain name. ↩

Upgrading a Nexus 6P from CyanogenMod 13 to LineageOS 14

2017-05-11T07:00:00+08:00

There is an experimental build of LineageOS 14 which can upgrade from CyanogenMod 13 but I thought this was a good opportunity to clear some of the cruft off my phone and start again.

Backing up the phone

There were only a few files on my phone I wanted to copy off. All my photos, contacts and calendar are synced with NextCloud so I already had them backed up.

But I wanted to get a copy of the seed values from Google Authenticator and I wanted to keep a copy of my text messages¹.

To pull out the Google Authenticator database you need to enable adb and root access².

To do this go into Setting > About Phone and tap on 'Buld Number' 7 times.

Then go back and go into Developer options.

Scroll down to root access and pick ADB only.

Scroll down to Debuguging and switch Android debugging on.

Next you need to install ADB onto your computer, for me on Arch Linux that was as simple as

sudo pacman -S android-tools

for Debian / Ubuntu that should be

sudo apt-get install android-tools-adb

Once you have adb installed you can connect your phone via the USB cable. You may need to accept a prompt on your phone to connect adb and trust the computer and the you can run

adb root
adb pull /data/data/com.google.android.apps.authenticator2/databases/databases authenticator.db

This will pull out your Google Authenticator database out. It's just a simple sqlite database and you can have a look through it.

sqlite3 authenticator.db

SQLite version 3.18.0 2017-03-28 18:48:43
Enter ".help" for usage hints.
sqlite> .tables
accounts          android_metadata
sqlite> select * from accounts;
1|Gmail|xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx|0|0|0
2|GitHub|xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx|0|0|0
3|Amazon Web Services|xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx|0|0|0

simiarly we can now pull out our old text messages database with.

adb pull /data/data/com.android.providers.telephony/databases/mmssms.db

and have a look through them

sqlite3 mmssms.db

Whipe the phone

We reboot the phone into TeamWin Recovery mode by holding the volume down key while booting. Then press the volume up and down to scroll to Recovery mode and click the power button.

Then pick Wipe. I personally did Advance Wipe and picked all the partitions. I don't think this was necessary but I think it's cleaner to start off with a compleatly blank phone.

Flash the new firmware

Next we flash the new firmware.

Reboot again, and hold the volume down key but this time wait at the menu where it says Reboot bootloader.

When I first ran fastboot devices I got an error message

fastboot devices
no permissions; see [http://developer.android.com/tools/device.html]    fastboot

But running it as root worked. I downloaded and flashed the vendor image, radio firmware and bootloader.

sudo fastboot flash vendor vendor-n2g47h.img
sudo fastboot flash radio radio-angler-angler-03.81.img
sudo fastboot flash bootloader bootloader-angler-angler-03.67.img

Install LineageOS

Now we reboot into into TeamWin Recovery mode.

Use adb to push a copy of lineage-14 onto the phone

adb push lineage-14.1-20170501-nightly-angler-signed.zip

and pick Install then select the zip file and follow the prompts.

Now we reboot then wait ... wait some more .... then after about a minute when your just starting to think "Oh Jeez I've bricked my phone" you will be greeted by the LineageOS boot screen.

I'm a bit of a hoarder with my data, storage is so cheap and it's so easy to keep it all forever. Signal has disappearing messages and it's a great idea, some conversations ephemeral but I just can't bring myself to delete my message history. ↩
Just incase it's not obvious enabling root access is a horible idea from a security point of view, I'd only do it when necessary and remove it again straight away. ↩

Inbox Zero

2017-05-04T07:00:00+08:00

There must be hundreds of diffrent strategies for dealing with the constant flood of requests that come in through out the day. My personal favorite for email is Inbox Zero. If you havn't come across it before you should go and check it out, it might not be the one that works for you but you should at least take a look.

The thing that really stuck for me was that your inbox shouldn't be a reminder system, most office suites already have two tools that do that better; your calender and a your task / to do list.

Instead of leaving an email that's otherwise dealt with in your inbox thinking "I must follow that up next week when Jo gets back from leave ... " archive the email and stick an appointment in your calendar for some time after Jo gets back that says "Follow up thing with Jo."

Installing Python 3.6 on Windows

2017-04-27T07:00:00+08:00

Installing Python on Windows is not that hard, but I want the 64 bit version and I want it installed system wide and neither of these are defaults so we need to change things a little.

First we go to python.org and navigate to Downloads > Windows

Then we need to download the 64-bit exceutable installer.

We run the installer, first tick "Add Python 3.6 to PATH" this makes life much easier because we can simply call python from the command line. Next click "Customize installation"

I install all the optional features (this is the default anyway) so we simply hit next.

In the Advanced Options tick "Install for all users" this will change the path from C:\Users\your.name\AppData\Local\Programs\Python\Python36 to C:\Program Files\Python36 which will make it system wide. Then hit Install

This will ask for elevated privlages and then run though installing and your done.

I understand that the Python installer probably defaults to installing the 32 bit version for just the local user for compatibility reasons. It's better to have defaults that will work for everyone, but for me I almost always install python to run things like scheduled tasks and system scripts so this is how I like my Python set up.

Spam Filtering

2017-04-20T07:00:00+08:00

Since I've started hosting my own email server I've been using spam assassin for my spam filtering. All in all it does a pretty good job of stopping spam but unfortuantly without the volume of mail Google has to train it's filter it will simply never be as good as what you can get from hosted email with one of the major providers.

Recently I've decided that instead of constantly trying to tweak my spam filtering rules, adding new real time black lists, and so on. I've simply turned spam assassin up to be very agressive and then started using a whitelist of people and domains that I receive email from regularly.

I know that it's not an approach that would scale past a couple of people, let alone a millions like the major providers have to deal with, but for me I rarely receive email from people I have not communicated with before and simply exporting my contacts list and whitelisting those addresses seems to be working pretty well.

Moving beyond PEP 8

2017-04-13T07:00:00+08:00

I recently watched Raymond Hettinger - Beyond PEP 8 -- Best practices for beautiful intelligible code - from PyCon 2015. It was an ammazing talk and I'd higly reccomend it to anyone wanting to write readable Python code.

One of the first things he tackled was the 79 character line limit. That's something that has always annoyed me, as I personally like long and descriptive variable names and if your already tabbed in four or eight spaces it dosn't always leave a lot of space for your code. I know I've been guilty of writing worse and less readable code just to make my code squeeze into the space available. Really if your code is PEP 8 compliant but harder to read then you have missed the point of PEP 8.

I feel very strongly about descriptive variable names, I've worked with software that was originaly written for Gupta SQL Base in 1992 when it didn't support column names longer than 18 character. This lead to some great names like PRJ_CON_RET_DAT, you might be able to work out what is but a variable like project_contract_returned_date is so much easier to understand. Some people have said long descriptive variable are hard to type, but I think people should be using an editor with autocomplete.

Raymond also tackes a few other ways to make your code more pythonic.

Calculating a base64 encoded sha256 sum of inline scripts for your content security policy

2017-04-06T07:00:00+08:00

A while ago I wrote a post on HTTP Security Headers and part of that invloved setting up a content security policy (CSP) and in that I say

I've done a SHA-256 hash of the script

and I just left it at that, simple right? Only now a it's little over a year later, I've changed my piwik domain and I need to change my inline script only I can't remember how I calculated the sum.

For those who have already have a CSP I'd recomend;

Open Chrome
Hit F12 to get the console
Load your page
Find the error message which helpfully contains exactly what you need to add to your CSP

So in my case chrome provided me with:

Either the 'unsafe-inline' keyword, a hash ('sha256-j69kMLNHErwf2Xyju05S+HrqhF6iQdmyWjxO2peCm10='), or a nonce ('nonce-...') is required to enable inline execution.

(emphasis mine)

Of course that's fine if you are ok with temporarily breaking your script but what if you want to calculate it before putting it on you your site?

My new inline script is:

<!-- Piwik -->
<script type="text/javascript">
  var _paq = _paq || [];
  _paq.push(['trackPageView']);
  _paq.push(['enableLinkTracking']);
  (function() {
    var u="//piwik.xo.tc/";
    _paq.push(['setTrackerUrl', u+'piwik.php']);
    _paq.push(['setSiteId', 2]);
    var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0];
    g.type='text/javascript'; g.async=true; g.defer=true; g.src=u+'piwik.js'; s.parentNode.insertBefore(g,s);
  })();
</script>
<noscript><p><img src="//piwik.xo.tc/piwik.php?idsite=2" style="border:0;" alt="" /></p></noscript>
<!-- End Piwik Code -->

Now we don't include the <script> tags but white space is significant so in my case I needed a line break (blank line) at the start because there is a linebreak just after the opeing <script> tag but I didn't need a blank line at the end. I saved a text file with the script in it and ran

openssl dgst -sha256 -binary piwik_script.txt | openssl enc -base64

which is based on the example from the W3C recommendation about CSPs.

Occam's razor

2017-03-30T07:00:00+08:00

I mentioned Occam's razor in a previous post and it's a philosophy I'm a huge fan of. Especially in information secuirty.

Often it's summed up as

"the simplest explanation is usually the correct one"

I think it's easy to get carried away with theories that could be posible rather than focus on the theories that are the most likely.

The HP Automated Storage Manager Server service terminated unexpectedly.

2017-03-23T07:00:00+08:00

About a week ago we had an issue with a HP X1600 G2 Network Storage System Server. After rebooting we got an error message in the event log roughly ever minute with Event ID 7031

The HP Automated Storage Manager Server service terminated unexpectedly. It has done this 1 time(s).  The following corrective action will be taken in 0 milliseconds: Restart the service.

followed by an informational log with Event ID 7036

The HP Automated Storage Manager Server service entered the running state.

and it kept looping through those two messages.

After some digging around we worked out that it was a USB hard drive causing the error. The hard drive had been plugged in for almost a month but it seems the version of HP Automated Storage Manager we were running couldn't start if there was a USB hard drive attached. Attaching the drive after the service has started doesn't cause issues which is why the error didn't crop up until after we rebooted the server.

Hopefully that helps someone else who is scratching their head trying to work out this issue.

Attribution is about more than just technical evidence

2017-03-16T07:00:00+08:00

There is a story that's been doing the rounds lately about malware that took control of microphones and uploaded over 600 GB of audio to its command and control. As others in the security industry have pointed out this is a great example of where we can make fairly confident guesses about the origin of the malware without even looking at the "Technical" evidence; Things like network logs, packet captures, infection vectors and reverse engineering the binaries to look for clues to trace the attacks back.

Instead, we can look at a more political angle. As a quick off the back of the envelope calculation if we had 600GB MP3 files at 128kbps¹ it would be more than 10,000 hours or over a year of audio. We can also see the computers infected were in the Ukraine, so we can say "Who would have the capacity and desire to listen to over 10,000 hours of Ukrainian conversations?"²

Sure it might not prove attribution that would stand up in a court of law to "Beyond a reasonable doubt" but it points very strongly in one direction.

I was once involved in a situation where a document had been leaked. Several people had access to that document and any of them could have leaked it, but we looked at when the document was leaked, who it was leaked to and who stood to gain from the leak. In the end, we had a pretty good idea about who had leaked the document, maybe not "beyond a reasonable doubt" level of confidence but enough that we were satisfied.

I think in a heavily technical field where some things can be boolean it's easy to overlook the more social and political aspects where things are not so definite.

I have no idea what codec or compression levels were being used, but 128kbps is a fairly average rate. ↩
While that's possible that siphoning off the audio could be misdirection. I think Occam's razor applies here and "the simplest explanation is usually the correct one". ↩

One hundred prisoners and a light bulb simulation

2017-03-09T07:00:00+08:00

This is a little off my usual topic of IT Security but it's something I've been thinking about a bit lately. I recently came across the "One hundred prisoners and a light bulb" riddle. It was posed as:

A group of 100 prisoners, all together in the prison dining area, are told that they will be all put in isolation cells and then will be interrogated one by one in a room containing a light with an on/off switch. The prisoners may communicate with one another by toggling the light-switch (and that is the only way in which they can communicate). The light is initially switched off. There is no fixed order of interrogation, or interval between interrogations, and the same prisoner will be interrogated again at any stage. When interrogated, a prisoner can either do nothing, or toggle the light-switch, or announce that all prisoners have been interrogated. If that announcement is true, the prisoners will (all) be set free, but if it is false, they will all be executed. While still in the dining room, and before the prisoners go to their isolation cells (forever), can the prisoners agree on a protocol that will set them free?

I think there are several version that all run along the same lines but with slightly tweaked wording.

The general solution to the puzzle is that;

All the prisoners decied to elect one prisoner as the leader.
When a prisoner is interrogated if the light is off and they have not switched it on before they will switch the light on. Otherwise they will leave the light unchanged.
Only the leader can switch the light off. After the leaer has switched the light off 99 times they know all other prisoners must have been interrogated.

This works and from a logic point of view is fairly elegant. However it seemed inefficient to me. I wanted to know how many interrogations before the prisoners are set free. I feel sure there is some mathematical way you could calculate the average but that's beyond me so I ~~kidnaped 100 people and locked them in my basement~~ wrote a Python script to simulate the problem.

#!/usr/bin/python3
# -*- coding: UTF-8 -*-
"""
A group of 100 prisoners, all together in the prison dining area, are told that
they will be all put in isolation cells and then will be interrogated one by
one in a room containing a light with an on/off switch. The prisoners may
communicate with one another by toggling the light-switch (and that is the
only way in which they can communicate). The light is initially switched off.

There is no fixed order of interrogation, or interval between interrogations,
and the same prisoner will be interrogated again at any stage. When
interrogated, a prisoner can either do nothing, or toggle the light-switch,
or announce that all prisoners have been interrogated. If that announcement is
true, the prisoners will (all) be set free, but if it is false, they will all
be executed.

While still in the dining room, and before the prisoners go to their isolation
cells (forever), can the prisoners agree on a protocol that will set them free?
"""
import random

light_bulb_on = False


class Prisoner():
    """The basic class there should be 100 of these"""

    def __init__(self):
        """Sets up the initial variables"""
        self.has_switched_on_light_bulb = False

    def interigation(self):
        """
        When the prisoner goes into the room, if the light is on they leave it
        otherwise if it's off and they have not yet switched it on they turn
        the light bulb on
        """
        global light_bulb_on

        if self.has_switched_on_light_bulb is False and light_bulb_on is False:
            light_bulb_on = True
            self.has_switched_on_light_bulb = True


class Leader(Prisoner):
    """
    Only the leader can switch the light bulb off. After they have swtiched the
    light bulb off 99 times, they know all prisoners have been interrogated.
    """

    def __init__(self):
        """Sets up the initial variables"""
        Prisoner.__init__(self)
        self.switch_off_count = 0

    def interigation(self, ):
        """
        When the leader gets in interrogated they can switch the light off
        """
        global light_bulb_on

        if light_bulb_on:
            light_bulb_on = False
            self.switch_off_count += 1

        if self.switch_off_count == 99:
            return "All prisoners have been interrogated"


def run_simulation():
    """
    Runs a simulation of the onehundred prisoners and a light bulb problem and
    returns the number of interigations before the prisoners are released.
    """
    # Add one leader and 99 prisoners
    number_of_interigations = 0
    responce = None

    prisoners = [Leader()]

    for _ in range(99):
        prisoners.append(Prisoner())

    while responce != "All prisoners have been interrogated":
        responce = random.choice(prisoners).interigation()
        number_of_interigations += 1

    return number_of_interigations


# Run the simulation 1000 times and print out the average number of
# interigations before the prisoners are released.
average = 0
for _ in range(1000):
    average += run_simulation()

print(average // 1000)

Making that script object oriented is compleate overkill but it was fun to write. I've made some assumptions here, mainly that the prisoners are interrogated in a random order and continuously until one of them says "All prisoners have been interrogated".

It usually takes around 10400 interigations before the prisoners are set free. I then started thinking about other issues like what if the interigations are not random. My sister sent me a link to a journal article that looks into all these posibilities. It's a fun little distraction for those who like logic puzzles.

Using the new(ish) Nextcloud updater

2017-03-02T07:00:00+08:00

I first started playing with ownCloud back in early 2012 with version 3, and started using it seriously in mid 2014 as my main tool for syncing my Calendar, Contacts and files having migrated away from Google's Calendar/Contacts/Drive.

But one of my biggest complaints was the update mechanism, or rather the lack of one. On September 29th 2016 I published a copy of my ownCloud update script nothing amazing but it did the job. In that post I said that

I've been looking at NextCloud, I haven't made the switch yet but if they introduce an automatic update mechanism that would be a big enough draw card for me to change.

I didn't realise it at the time but also on the 29th of September 2016 Nextcloud annoucned a new updater mechanism. Not fully automatic like wordpress security updates, but a good step in the right directon.

Early in 2017 I decided to take the plunge and migrate across to Nextcloud and I've just done my first in browser upgrade, from 11.0.1 to 11.0.2.

It started with a notificaton in the browser and also on the desktop client.

I went into the admin panel

Under the version section I clicked Open updater

Then simply hit Start update

It ran through all of the steps in a couple of minutes then I picked "No" to exit maintenance mode.

Then I returned back to the home page to finish the database upgrade

From there, the update was the same as before; click on "Start update" to start upgrading the database

After the upgrade continue on to Nextcloud and re-enable any 3rd party apps that have been disabled.

The overall process was very easy and felt much more user friendly than SSHing in and running my bash script.

Using ssh config to save settings and make your life easier

2017-02-23T07:00:00+08:00

SSH is an amazing tool, I use it all the time and not just for logging into remote computers, but also to create tunnels, copy files and access git repositories.

But I don't want to have to remember fidily commands something like ssh I2P_Router is so much nicer than something like

ssh -i .ssh/Michael-Van-Delft.id_rsa -L 7657:localhost:7657 -L 4444:localhost:4444 -L 6668:localhost:6668 i2p_user@example.com:2233

I keep my ~/.ssh folder synced as a git repository, that I sync over https with gogs. For all other git repositories I use ssh, but https solves the bootstap problem where I setup a new computer and want to download my ssh settings.

Below is an example of my ~/.ssh/config file with just a few changes.

# Override /etc/ssh_config
# This is a potential issue where someone who can read your ~/.ssh/known_hosts
# can see what servers you have SSHed into. However I'm not concerned by that.
# It makes it much easier to just delete one line from known_hosts when a
# server changes key.

HashKnownHosts no

###############################################################################
# Servers                                                                     #
###############################################################################
Host example.com www.example.com brand.example.com
    IdentityFile ~/.ssh/Michael-Van-Delft.id_rsa
    User michael

# Port forwarding for I2P. Simply run `ssh I2P_Router`
# then browse to http://localhost:7657/
Host I2P_Router
    HostName example.net
    IdentityFile ~/.ssh/Michael-Van-Delft.id_rsa
    User i2p_user
    LocalForward 7657 localhost:7657
    LocalForward 4444 localhost:4444
    LocalForward 6668 localhost:6668

# Example of a local server with IPv6 only
Host zilean
    IdentityFile ~/.ssh/pi@zilean.example.com.id_rsa
    User pi
    AddressFamily inet6
    HostName 2001:0db8:6101:cc01::7

###############################################################################
# Git and Service accounts                                                    #
###############################################################################
Host github.com
    HostName github.com
    IdentityFile ~/.ssh/git@github.com.id_rsa
    User git
    IdentitiesOnly yes

# This is good if you have a server you ssh into (like example.com from the top
# entry) where you want to use diffrent credentials for git as you do when you
# SSH in normaly.
#
# To clone a repository simply run
# git clone gogs:Michael/exotic-security.git
Host gogs
     HostName example.com
     IdentityFile ~/.ssh/gogs.id_ed25519
     User git
     IdentitiesOnly yes

Simple postgres basics

2017-02-16T07:00:00+08:00

This is another one of those things I've done all this several times before, but need to keep looking up the documentation.

To login to PostgreSQL after a fresh installation¹ you need to change to the postgres user and run psql (no password needed, it uses Peer Authentication)

sudo -u postgres psql

from there you can create a user and access it as you normally would.

createuser -P -s -e michael

to connect to a database use

\c database_name

to list the tables run

\dt

and to make the output format expanded so it fits on a small screen simply switch

\x on

In my case that's usually on Debian or Ubuntu but it should be the same on most Linux distros. ↩

My IP Tables script example

2017-02-09T07:00:00+08:00

Below is an example of the IP Tables script I use on many of my servers. The names and IP addresses have been changed to reserved addresses and obviously it needs to be tweaked each time for relevent rules.

#!/bin/bash

# This script is symlinked to /etc/network/if-pre-up.d/firewall-rules
# ln -s /home/michael/firewall-rules.sh /etc/network/if-pre-up.d/firewall-rules

################################################################################
# IPv4 Rules
################################################################################

# Networks
MichaelHome="198.51.100.122/32"
MichaelHomeV6="2001:db8:62F8:cc01::0/64"
TienHome="203.0.113.94/32"
WorkNetwork="192.0.2.0/24"

function GeneralRules {
        #start and flush
        $IPTABLES -F
        $IPTABLES -t nat -F
        $IPTABLES -X
        $IPTABLES -P FORWARD DROP
        $IPTABLES -P INPUT   DROP
        $IPTABLES -P OUTPUT  ACCEPT

        #Ping, Trace Route, etc...
        $IPTABLES -A INPUT -p icmp -j ACCEPT

        #Mail - SMTP, SMTPS, IMAP and IMAPS
        $IPTABLES -A INPUT -p tcp --dport 25 -j ACCEPT #SMTP
        $IPTABLES -A INPUT -p tcp --dport 465 -j ACCEPT #SMTPS
        $IPTABLES -A INPUT -p tcp --dport 587 -j ACCEPT #SMTP Submission
        $IPTABLES -A INPUT -p tcp --dport 143 -j ACCEPT #IMAP
        $IPTABLES -A INPUT -p tcp --dport 993 -j ACCEPT #IMAPS
        $IPTABLES -A INPUT -p tcp --dport 4190 -j ACCEPT # dovecot-sieve set mail filter settings.

        #HTTP[S] traffic
        $IPTABLES -A INPUT -p tcp --dport 80 -j ACCEPT
        $IPTABLES -A INPUT -p tcp --dport 443 -j ACCEPT

        # i2p
        $IPTABLES -A INPUT -p tcp --dport 21546 -j ACCEPT
        $IPTABLES -A INPUT -p udp --dport 21546 -j ACCEPT

        # zeronet
        $IPTABLES -A INPUT -p tcp --dport 15441 -j ACCEPT

        #Allow Establishted Sessions
        $IPTABLES -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

        #loopback
        $IPTABLES -A INPUT  -i lo -j ACCEPT
        $IPTABLES -A OUTPUT -o lo -j ACCEPT

}
function IPv4Rules {
        # SSH and Mosh
        $IPTABLES \
            -A INPUT \
            -p tcp \
            -s $MichaelHome,$TienHome,$WorkNetwork \
            --dport 22 \
            -j ACCEPT
        $IPTABLES \
            -A INPUT \
            -p udp \
            -s $MichaelHome,$TienHome,$WorkNetwork \
            --dport 60000:60010 \
            -j ACCEPT
}

function IPv6Rules {
        # SSH and Mosh
        $IPTABLES \
            -A INPUT \
            -p tcp \
            -s $MichaelHomeV6 \
            --dport 22 \
            -j ACCEPT
        $IPTABLES \
            -A INPUT \
            -p udp \
            -s $MichaelHomeV6 \
            --dport 60000:60010 \
            -j ACCEPT
}

#Run general rules for both IPv4 and IPv6
IPTABLES=/sbin/iptables
GeneralRules
IPv4Rules

IPTABLES=/sbin/ip6tables
GeneralRules
IPv6Rules

#DHCP
$IPTABLES -A INPUT -p udp --dport 546 -j ACCEPT
$IPTABLES -A INPUT -p icmpv6 -j ACCEPT

pip changing from pep8 to pycodestyle

2017-02-02T07:00:00+08:00

I recently updated one of the packages in Atom that was called linter-pep8 to version 2.0 which was renamed to linter-pycodestyle. This is because PEP8 the package was renamed to pycodestyle to reduse confusion between the package and the PEP8 the specification.

However after I opened Atom I got an error message Error: spawn pycodestyle ENOENT

because I hadn't upgraded the python package. As I wasn't using PEP8 for anything else I uninstalled it and installed pycodestyle. On windows I'd installed Python 3.6 x64 for all users so it was Python was installed C:\Program Files\Python36\

"C:\Program Files\Python36\Scripts\pip.exe" uninstall pep8
"C:\Program Files\Python36\Scripts\pip.exe" install pycodestyle

on Linux¹ pip was in my PATH environment varable so I simply ran

sudo pip uninstall pep8
sudo pip install pycodestyle

And that fixed up my issues.

In my case that was Arch Linux and Debian Jessie. ↩

Failing Loudly

2017-01-26T07:00:00+08:00

There is a concept in IT called 'failing loudly' as opposed to 'failing silently'. The idea is when something goes wrong it should be obvious and generally everything should come to a halt instead of trying to carry on with errors.

An example of this is running a REST API and only opening port 443, but leaving port 80 is closed. Connections are either secure or don't work at all.

There was a change with systemd where if the /etc/fstab files had errors in it, the system would hang at boot forever until some sort of user input fixed the issues. The previous behaviour was to simply show an error while booting but continue on regardless. The systemd argument was that it's better not to boot at all than to boot into a broken state, such as with a hard drive missing and potentially lose data.

As with every design approach it has it's place, it's not always the appropriate way to do things. It comes down to what you want to prioritizes. But I think it's very appropriate for things which need good security.

Expiry dates on smart phones and other IoT devices

2017-01-19T07:00:00+08:00

A while ago someone¹ suggested the idea of putting an expiry date on smart phones. The idea was that when manufacturing a device the company would have to commit to pushing out fixes to any CVEs that come up until the given date. So when buying a phone there would be an expiry date printed on the packageing and consumer could be sure of reciving a supported product until that time.

After the Mirai botnet struck there was a lot of discussion around what to do about the Internet of Things (IoT) threat. Bruce Schneier wrote that it was

a market failure that can't get fixed on its own.

and that it needed some sort of government intervention to fix. I tend to agree with his analysis, there is little to no incentive for vendors to fix the bugs in some internet connected smart toaster. Most consumers don't care if their $20 toaster has been hacked and used to DDoS some website, so long as it sill makes toast. And most vendors of IoT stuff don't have long product cycles and certanly don't budget the time and resources to fix things two years after they have been sold.

The aproach I'd take² to fixing the IoT threat would be to introduce manditory expiry dates for internet connected things. This wouldn't mean consumers couldn't continue to use them after the expiry date, just that the manufacturers must fix issues with products that have not expired and vendors can't sell expired items. It could be on some sort of sliding scale so things like internet connected washing machines might be 5 years while phones might only be 2 years. A bit like a manufacturer's warranty.

I'd introduce some sort of certification, a minimum security standard that devices need to conform to. This would be pretty simple check box security but it would be good base line. Things like the device must have some sort of automatic update process so that when things do go wrong, they can be fixed. And the update process should check the updates are signed.

I'd also heavily push some standard environments, things like Raspberry Pi's running Raspbian and Windows 10 for IoT³. This would make certification easier because the base environment could already be certified and could make best practice easier and shooting yourself in the foot harder.

After a fair amount of searching I still can't find the original source but I'm pretty sure it was a comment on an LWN article about a horrible android bug (possibly libstagefright) where I first came across the idea. ↩
Let's just pretend we live in a fantasy world here where governments could move quickly and cooperate, and import and export regulations could actually be applied to things like $13 internet connected light bulbs for sale on eBay. ↩
I think diversity is important and I'd like to see at least 3 or 4 base platforms. If nothing else so you don't get one bug that just ripps through all devices. ↩

Submission to the Attorney-general's Department - Access to telecommunications data in civil proceedings

2017-01-12T00:00:00+08:00

On the 21st of December 2016, the Attorney-general's Department requested submissions regarding the use of telecommunications data held by a service provider solely for the purpose of complying with the mandatory data retention regime in civil litigation. The original closure date for submission was the 13th of January but it's been moved back to Friday the 27th January.

The following is an open letter I'm making as a Submission to Attorney-general's Department, I'd encourage others to use it as a template and make their own submissions. Also avalible as a LibreOffice and a pdf versions.

Retained data in civil proceedings consultation
Communications Security Branch
Attorney-General's Department
3-5 National Circuit
BARTON ACT 2600

Submission against the use of telecommunications data held by a service provider solely for the purpose of complying with the mandatory data retention regime in any civil litigation

When mandatory metadata retention laws were first announced several people and high profile organisations raised concerns about the storage and use of this incredible amount of very personal data being kept on all Australians. However the laws were ushered through under the guise of national security. It was claimed that stronger powers were needed to protect Australia from terrorism¹ and that this huge expansion of law enforcement capabilities would be used by intelligence agencies to fight Islamic State².

The metadata facts sheet³ released by the Attorney-general's Department says that

Metadata is vital to nearly every counter-terrorism, organised crime, counter-espionage and cyber-security investigation. It is used in almost every serious criminal investigation, including murder, sexual assault, child exploitation and kidnapping.

However from the day mandatory data retention was introduced it was feared that this information, described in an opinion piece by George Brandis titled "One more anti-terror tool" as being "vital to investigate terrorism and organised crime."² would instead be subject to mission creep. Many predicted that metadata kept solely for the purpose of complying with the mandatory data retention regime would go from a tool only to be used in "serious criminal investigation" to a source of information for petty crimes and civil litigation.

In the Consultation Paper it is mentioned that

In the course of the Committee’s inquiry into the Bill, a number of submissions expressed concerns that retained telecommunications data would be able to be accessed by parties to civil proceedings.

In its Advisory Report on the Telecommunications (Interception and Access) Amendment (Data Retention) Bill 2014, the Committee recommended that the Bill ‘be amended to prohibit civil litigants from being able to access telecommunications data that is held by a service provider solely for the purpose of complying with the mandatory data retention regime.’

I find it particularly worrying that the question on the Data Retention FAQs that talked about use in copyright enforcement has now been removed, previously⁴ it said:

Will data retention be used for copyright enforcement?

The Telecommunications (Interception and Access) Act 1979 only allows access for limited purposes, such as criminal law enforcement matters. Breach of copyright is generally a civil law wrong. The Act will preclude access to telecommunications data retained solely for the purpose of complying with the mandatory data retention scheme for the purposes of civil litigation.

and I fear that a tool which was originally introduced to fight terrorism will now become a tool of large private media organisation perusing copyright violations.

In regards to the question 3;

Are there particular kinds of civil proceedings or circumstances in which the prohibition in section 280(1B) of the Telecommunications Act 1997 should not apply?

I believe the answer should be a strong and firm "No, the prohibition in section 280(1B) of the Telecommunications Act 1997 should apply to all types of civil proceedings". We should be looking to strengthen our controls and protections around this data not to weaken them.

ABC the 730 Report 'Democracies must be on front foot' says George Brandis as Government prepares new laws ↩
The Australian One more anti-terror tool ↩↩
The Attorney-general's Department Data retention facts ↩
Snapshot of the FAQs on archive.org from 28/04/2016 and the current version ↩

Thinking about how to defend against the PoisonTap

2017-01-05T07:00:00+08:00

Recently the Samy Kamkar has come out with a device called a PoisonTap, a few months before that Mubix was talking about getting credentials from a locked computer with the LAN Turtle.

Both these attacks exploit the same underlying issue which is that most operating systems (Windows, Linux¹ and OSX) will automatically trust a USB network when it's attached and start sending data over it.

I've been thinking a lot about how we as the IT Security Community can defend against these sorts of attacks.

The most obvious idea that springs to mind is to issue the user with some sort of popup "New network detected, do you want to connect?" but there are a few issues with that.

The first is that it's a horrible user experience (UX) because 99.9% of the time the answer will be "Yes" ... "Why do you think I plugged in my usb 4g dongle if I didn't want to use it!?".

The second is that sometimes you need the network to start working before you can login. A few years ago I worked at a high school we used RADIUS to secure our WiFi. Students could connect with their domain credentials. We had shared laptops in the school library but the laptops couldn't authenticate with the RADIUS server until students had logged in, but students couldn't login to the laptops without network. This will likely only get worse, with devices like Chromebooks and Windows 10 pushing Microsoft accounts pretty hard.

The defences that PoisonTap jokingly suggest for desktop security are funny but impractical such as

Adding cement to your USB and Thunderbolt ports can be effective

In the end I don't really think there is any good client side defence for these sorts of attacks. Instead I think it needs to be at the protocol level, we need to bake security in by default. Things saying browsers vendors saying we will only support HTTP/2 if it's encrypted.

We should demand encryption in any new protocol and systems susceptible to passive monitoring should be treated as a vulnerable and rejected. It might be a long an painful journey but I can imagine an internet where all communications are secure by default.

I always though it was disappointing that IPv6 didn't make encryption mandatory it would have been great to have security built right in at the Internet layer.

There are hundreds of distributions but, when I say "Linux" I mean mainstream distributions like Debian / Ubuntu / Red Hat / Fedora with their default settings. ↩

Fixing no valid mx hosts found

2016-12-29T07:00:00+08:00

I've been hosting my own email on this domain for just over a year now and I hadn't noticed any problems until a couple of days ago a German friend of mine using GMX tired to send me an email and it bounced back with the error message "no valid mx hosts found".

It turns out that according to RFC 2181

a MX resource record must not be an alias.

I had mail.xo.tc setup as my mx record

And then because everything is running off this one server I had mail as a CNAME.

I deleted the CNAME and added in an A and AAAA record and that fixed the issue.

Now I can receive emails from gmx.de and I'm RFC compliant.

Upgrading from Piwik 2.17.1 to Piwik 3.0.0

2016-12-22T07:00:00+08:00

Piwik have just announced the release of Piwik 3.0 and I was excited to try it out. I've been running Piwik on this site for just over a year. I like Piwik because it allows me to run analytics¹ on this site while respecting users privacy, giving users the option to opt-out² of tracking and it means I don't share their data with a 3rd party like Google.

The upgrade was fairly seamless.

When I logged in there was a notification saying an update was available.

I clicked on the update link to update and selected "Update Automatically"

After about 10 seconds I got a message saying the update was successful

Then I had to upgrade the database, as this is a very low traffic site I decided to upgrade in the browser

The update finished and I continued on to piwki

Only when I reloaded the page I was getting 500 server errors

Looking in /var/log/apache2/error.log I saw a number of errors saying:

/var/www/html/piwik/plugins/.htaccess: Options not allowed here

I found that the .htaccess file in the plugins directory had a line at the end

Options -Indexes

That line stops people getting a directory listing of the files in the plugins folder, so it's an important security feature.

I edited my apache2 config and changed my AllowOverride setting from

<Directory /var/www/html/piwik>
        AllowOverride FileInfo Limit AuthConfig
</Directory>

<Directory /var/www/html/piwik>
        AllowOverride FileInfo Limit AuthConfig Options=Indexes
</Directory>

After that Piwik loaded up with no erros.

I've had a bit of a play with it and I think the new dashboard looks nice.

One of the features I've been tracking and looking forward it is the change from md5 hashes to bcrypt so it's great to see that's landed in the 3.0 release.

If pushed, I'd have to begrudgingly admit that it's more about self validation than any technical usefulness of the data. ↩
It would be better if all web analytics were opt-in, but that's not how the world seems to work. ↩

Is patching still the best defence

2016-12-15T07:00:00+08:00

One of the nice things about the ASD's Strategies to mitigate Targed Cyber Intrusoions is that they rank them by effectiveness. Saying that their top four would prevent 85% of intrusions.

Their top four are:

Application Whitelisting.
Patching Applications.
Patching Operating System Vulnerabilities.
Restrict Administrative privileges.

Patching is two out of the top four recommendations and has long viewed by many IT Security professionals, my self included as one of the easiest things to do that gets you the best bang for your buck.

I was at a meeting recently where someone said an exploit had not been used against their network in about 6 months. The implication was that a huge percentage of malware these days is delivered as an .exe in a zip file from an email claiming to be a traffic infringement or something similar. Suggesting that patching is no longer the easiest win for IT Security.

I'm not entirely convinced that exploits are no longer being used, I think there are plenty of hacked sites and malvertising campaigns that take advantage of unpatched browsers or out of date applications like flash. But I can see a bit of a shift from using exploits to infect computers to simply sending a trojan or a phishing email and relying on tricking users.

Tracking a spam campagn

2016-12-08T07:00:00+08:00

At work yesterday we got a couple of phishing email claiming to be traffic infringements, nothing too remarkable about that. But interestingly this time the links in the emails to Bitly, a url shortener service that redirects traffic instead of going directly to some hacked site hosting malware.

With Bitly URLs you can simply put a + sign on the end of any link it will take you to a page of statistics rather than redirect you. The URLs from the phishing emails (with a plus sign added) were:

https://bitly.com/2h3aFul+

and https://bitly.com/2gh3gXg+

We can see the first time either link was followed was around 21:00UTC wich is 05:00 AWST (Western Australia time) and it dies off pretty quickly suggesting that these campaigns move from one URL to another very quicly rather than spamming out the same URL all day¹. It also shows that most traffic is from Australia which you would expect given that the was claiming to be an infringement form the West Australia Police.

Most of the traffic is direct access, this not surprising seeing that people are coming from email rather than another source such as twitter. Although there is a fair amount coming from localhost:5272. I’m not sure what that is, but a quick google search suggest it’s Xeams spam filter is following links in emails to check if they are malicious.

The spoofed address this came from was infringement@data.gov.au, I'd be guessing they picked that one because it looks almost legitimate and data.gov.au doesn't have dmarc or even an SPF record². I was also surprised to find that in the footer of the email, the links were not malicious. There was one to the about West Australian Police and it really pointed to the about West Australian Police page. It was just in the body of the email that there were malicious links.

It's posible that the spammers have setup their campaign to send emails early in the morning for their target timezone. I remember hearing from a legitimate (double opt-in) email marketing group that first thing in the morning was the most effective time to send emails because it will be a the top of people inbox as they are having their morning coffee. ↩
I was really surprised to find that, I know a couple of the folks that helped set it up and they were pretty switched on types. ↩

Automatic Updates for Debian

2016-12-01T07:00:00+08:00

Good security is about defence in depth, layers of security. There is no one thing that will make you secure but one of the easiest things to do that gets you the best bang for your buck is patching your software.

On windows this is called Automatic Updates, in Debian it's called Unattended Upgrades but it's essentially the same thing. There is an Unattended Upgrades page on the Debian wiki that is pretty good. Enabling updates basicly boils down to:

sudo apt-get install unattended-upgrades apt-listchanges
sudo dpkg-reconfigure -plow unattended-upgrades
vim /etc/apt/apt.conf.d/50unattended-upgrades
# Edit line 71 to send emails to a monitored address

If your current update strategy is to SSH into boxes and run sudo apt-get update && sudo apt-get dist-upgrade whenever you remember then you should look automating it with unattended upgrades. Of course a full dev > test > production patch cycle is best for large mission critical things but for small setups like the one box that runs this website unattended upgrades are perfect.

The struggle with apathy

2016-11-24T07:00:00+08:00

I like many people struggle¹ with apathy, it's not that I'm depressed or that I don't find enjoyment in doing things. I still get that hacker like thirst for knowledge, that need to solve a problem or to understand what makes something tick. But sometimes I think it would be so easy to spend a whole day on the couch just watching weak sitcoms and eating Cheezels, I could pass a lot of time watching funny YouTube videos.

One of the reasons I update this blog with a new post published at the same time every week is that I've made a commitment to my self, a schedule that I can stick to, a dead line that I need to meet. If I just updated this blog on an adhoc basis whenever I was in the mood I'd probably have about 4 post on here. It's not that I don't enjoy writing posts but that without some self imposed pressure I'd never get done.

Generally when I see self help style things that are meant to be motivational they seem sickly sweet to me and put my right off. If you search positive motivational on Google and seeing things like "Choosing to be positive and having a grateful attitude is going to determine how you're going to live your life." and that makes you feel good, and it helps you achieve your goals that's awesome! Good for you.²

But for me it makes me feel a little nauseous. I don't really know how to explain, but the closest I can get is: It's like it's too happy to the point where it feels fake, and fake happiness feels worse than just feeling neutral.

Yan Zhu wrote an good post about apathy.

But there are some motivational things that I like;

Matt Grey and Tom Scott recently did a bit on how they manage to do so much stuff, the whole video is pretty decent but there is a bit where Tom says "Find someone who you can't let down". I like that, it could be a friend, family or or whatever but if you feel like you will disappoint or inconvenience someone you care about by not doing something they you're more likely to make the effort to do it.

I like goals that are very specific, attainable and very easy to evaluate. Now I know that has a faint whiff of bovine manure, but what I mean by that is don't say "I'd like to learn to use Metasploit". That's way too open ended and it won't happen. What does it mean to 'use'? to what level of proficiency? by when? how will you know when you can 'use Metasploit' well enough. Instead be very, very specific, say "I'm going to finish one chapter of this book on Metasploit every week for the next 17 weeks, I'll do it by doing an hours study every Sunday between 10:00 and 11:00 and if something comes up I'll move my hours study to Wednesday nights." That's much more achievable and you will know if you have failed or succeeded.

Struggle may be too strong a word, I was going to call this post 'The war on apathy' which is a much snappier title. But I have a strong objection to people declaring 'war' on everything. So I spent some time on thesaurus.com and eventually gave up and went with struggle because I couldn't be bothered looking any more. The irony of that is not lost on me. ↩
Incase your unsure, I mean that sincerely. We are not all the same and what works for one person doesn't for another. If those motivational posters are for you, then that's great. ↩

How to tunnel data over DNS

2016-11-17T07:00:00+08:00

A while ago I wrote a post on tunneling data over DNS that was a technical explanation of what's going on. This post is a tutorial on how to setup a DNS tunnel with iodine.

I'm using DNS Made Easy as my main DNS provider, Debian on an EC2 for the DNS server, and Arch Linux and Windows as the client. I haven't tried this on other setups but it should be more or less the same.

Setting up the server

Start a t2.micro EC2¹ running Debian, updated all the packages and reboot².

sudo apt-get update && sudo apt-get dist-upgrade && sudo reboot

Then install iodine

sudo apt-get install iodine

Setup the domain name to use for the tunnel, in my case I used tunnel.xo.tc. I created a sub-domain and delegated it to a DNS servers called tunnel-ns1.xo.tc.

In DNS Made easy go to NS Records and hit the Add button.

Next we need to setup the A record for the name server we have specified.

Then on the EC2 server

sudo iodined -f 10.73.72.1 -c tunnel.xo.tc

-f Keeps iodined running on the forground, it's not nessacery but it makes it easier to stop and start.
10.73.72.1 is the local network that iodine will create. Use an address that is not on your LAN.
-c Disables checking of the client IP address, you will only need it if your DNS queries are getting routed through a cluster of DNS servers and so your traffic will be coming from diffrent IP addresses.
tunnel.xo.tc is the domain to use as a tunnel.

Check your server is working with iodine check tool

Arch Linux Client

On the client (Arch Linux)

sudo pacman -S iodine
sudo iodine -f tunnel.xo.tc

Now if you run ip addr you should see a new network connection

[michael@ezreal ~]$ sudo ip addr
# ...
4: dns0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1130 qdisc fq_codel state UNKNOWN group default qlen 500
    link/none
    inet 10.73.72.2/27 scope global dns0
       valid_lft forever preferred_lft forever
[michael@ezreal ~]$ ping -c 4 10.73.72.1
PING 10.73.72.1 (10.73.72.1) 56(84) bytes of data.
64 bytes from 10.73.72.1: icmp_seq=1 ttl=64 time=346 ms
64 bytes from 10.73.72.1: icmp_seq=2 ttl=64 time=334 ms
64 bytes from 10.73.72.1: icmp_seq=3 ttl=64 time=346 ms
64 bytes from 10.73.72.1: icmp_seq=4 ttl=64 time=338 ms

--- 10.73.72.1 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3001ms
rtt min/avg/max/mdev = 334.381/341.482/346.673/5.310 ms

Congratulations, you now have a tunnel through DNS.

Windows Client

First we need to install the Install the TAP32 driver. To do this download the OpenVPN installer³ and when you get to Choose Components step, you only need to pick TAP Virtual Ethernet Adapter.

The rest is more or less the same as Linux, download the latest binary, extract them, open a command prompt as administrator and run iodine c:\Users\Michael\Downloads\iodine-0.7.0-windows\64bit\iodine.exe -f tunnel.xo.tc

Making it a service

Of course if you're planning on using it from a hotel WiFi for example you might not be able to SSH in and start iodine so you will want your tunnel available all the time.

sudo nano /etc/default/iodine

Setup your iodine config⁴

# Default settings for iodine. This file is sourced from
# /etc/init.d/iodined
START_IODINED="true"
IODINED_ARGS="10.73.72.1 -c tunnel.xo.tc"
IODINED_PASSWORD="SjLYBVAI4HnaF6TN6oryN7r2"

sudo systemctl enable iodined.service
sudo systemctl restart iodined.service

Encryption and routing

Now you have a DNS tunnel between you and your server, but it doesn't mean that all your traffic will magically flow through it, nor is your traffic private⁵. The recommended way is to either setup a VPN or SSH Tunnel⁶.

On Linux it's pretty simple ssh -D 8080 admin@10.73.72.1 -i aws-key.pem

On Windows it's pretty much the same, except we will use Putty and under Connection > SSH > Tunnels and Dynamic port forwarding on port 8080.

)

Then in Firefox go to Options > Advanced > Network > Connection Settings > Manual proxy configuration and enter the SOCKS proxy details.

I found the network to be painfully slow, but it's a fun little experment.

Also if you are using Amazon, make sure you open ports TCP 22, TCP 53 and UDP 53 in the security groups settings. ↩
A reboot is not strictly necessary, but when I ran the updates it installed a new kernel so I wanted to reboot for the kernel update. ↩
The documentation says it needs to be the 32 bit version of OpenVPN but I used the 64 bit version and it worked fine. ↩
To generate a password I recommend sudo dd if=/dev/random bs=1 count=18 2>/dev/null | base64. ↩
Tunneling data through DNS might be stealthy but iodine does not provide encryption be default. ↩
Yes a tunnel with in a tunnel. ↩

You should try to outrun the bear

2016-11-10T07:00:00+08:00

If you have worked in IT Security for a while you will probably have heard the old saying;

"You don't have to outrun the bear you only have to outrun the other bloke"¹

I've heard it several times and it annoys me because it's almost always used to defend doing a half-baked job of something. Things like using WEP because the neighbour's WiFi is unsecured.

Now I would never say things need to be perfect you're better off with something that works and is good than waiting for the perfect solution to be built. But if you're going to do something you should at least try to do things properly.

You should at the very least try to outrun the bear.

Or some variation on the theme. The one I originally heard was "the tiger" which I think sounds better, but Google suggest that "the bear" is more popular. ↩

Seting up Matrix Synapse and Riot on Debian 8 Jessie

2016-11-03T07:00:00+08:00

My partner is going over seas and wanted to be able to make video calls to me back in Australia. Unfortunately because I use F-Droid and don't have the Google Play store on my phone¹ I don't have WhatsApp or Viber or Facebook Messanger or... But I recently came across Riot.im a messaging app with the concept rooms like IRC or Slack and that can do one to one video calls.

So I decided to setup my own Matrix / Synapse server at home on Debian 8 Jesse.

I went with the packages rather than installing from source because I like the idea of an easy sudo apt-get update && sudo apt-get dist-upgrade to keep everything up to date.

Add the matrix-synapse signing key

wget https://matrix.org/packages/debian/repo-key.asc
sudo apt-key add repo-key.asc
rm repo-key.asc

Edit your sources.list

sudo vim /etc/apt/sources.list.d/synapse.list

add

deb http://matrix.org/packages/debian/ jessie main
deb-src http://matrix.org/packages/debian/ jessie main

Refresh your sources and install

sudo apt-get update
sudo apt-get install matrix-synapse

The install ask you for a host name, and if it can report anonymized statistics back home.

Configure

That's pretty much it, most of the defaults are ok, I'd recomend reading through homeserver.yaml anyway.

sudo vim /etc/matrix-synapse/homeserver.yaml

I enabled registration, then removed it once I'd signed up (line 294).

# Enable registration for new users.
enable_registration: True

On this server I'm not using Let's Encrypt yet so I copied my SSL certificates over the top of homeserver.tls.crt and homeserver.tls.key

And started the server.

sudo systemctl enable matrix-synapse.service
sudo systemctl statrt matrix-synapse.service

DNS Entry

I setup a DNS entry to tell federated servers what port to connect on. For me that was just entering a SRV record in DNS Made Easy.

The exact steps steps will be a little different depending on your DNS provider.

Apache2

Lastly I setup Apache to proxy /_matrix from port 443 to port 8008, below is part of my Apache2 config from /etc/apache2/sites-enabled/000-default.conf but the important bit is after # Matrix Synapse

<VirtualHost *:443>
    # Host settings
    ServerName hybr.id.au

    # SSL Settings
    SSLEngine on
    SSLOptions +StrictRequire
    SSLHonorCipherOrder on

    # Remove all, Add back only TLS1.2
    SSLProtocol -ALL +TLSv1.2

    # A fine selection of the choicest ciphers
    SSLCipherSuite -ALL:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256

    SSLCertificateFile /etc/ssl/custom/example.com.crt
    SSLCertificateChainFile /etc/ssl/custom/1_intermediate.crt
    SSLCertificateKeyFile /etc/ssl/private/exmple.com.key

    Header always add Strict-Transport-Security "max-age=31536000"   

    Header always set Public-Key-Pins "pin-sha256=\"f5uthPZ21VOlA6Bye2yvoe+6a/h9fKRK27SdFt43XHQ=\"; pin-sha256=\"ATwpV5xzLfkVs631iympx7q+JlvRePMgTcvFG7x3Eeo=\"; max-age=5184000; includeSubDomains"

    ServerAdmin webmaster@example.com
    DocumentRoot /var/www/html

    # Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
    # error, crit, alert, emerg.
    # It is also possible to configure the loglevel for particular
    # modules, e.g.
    #LogLevel info ssl:warn

    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined

    ################################################################################    
    # Matrix Synapse  
    ################################################################################    

    ProxyPass /_matrix http://127.0.0.1:8008/_matrix
    ProxyPassReverse /_matrix http://127.0.0.1:8008/_matrix

</VirtualHost>

Then you can go to https://riot.im/app/#/register pick the custom server radio button and away you go.

I've said it before, as this is a security blog I should point out that I use F-Droid and CyanogenMod for Open Source philosophical reasons and not for security reasons. If you want a secure Android phone get a modern Nexus phone or the Google Pixel and stick on the stock ROM with all the Google updates. ↩

Backing up a remote server with rsync

2016-10-27T07:00:00+08:00

I've got a couple of VPS boxes over at RansomIT including the server that run this site¹. One of those boxes is a cheap little VPS with 512MB of RAM that costs me $5 a month. I need to back up this box, my usual go to for personal servers is Crashplan but Crashplan needs 1GB of RAM.

So I thought, why not sync² the contents of the small server over to more powerful server (with 8GB of RAM) that can run Crashplan.

Create a backup user on the source server

I'm going to create a 'backups' account on the source server, add an SSH key and add the backups account into the sudoers group³.

sudo useradd --system --shell /bin/bash --home-dir /home/backups --create-home backups
sudo su backups
cd
ssh-keygen
# Accept the defaults
mv id_rsa.pub authorized_keys
less ~/.ssh/id_rsa
# Copy the private key and press Q to quit
exit
sudo visudo

Add to the sudoers file

# Allow backups to run rsync as root without a password
backups ALL=NOPASSWD:/usr/bin/rsync

Create our backups script on the destinaton server

Copy the ssh private key (id_rsa) to the destination server.

mkdir backups
cd backups
vim backups.id_rsa
sudo chown root:root backups.id_rsa
sudo chmod 400 backups.id_rsa
# SSH into the source server, this is both so we get the servers host key
# added and also a a bit of a 'hello world' sanity check.
sudo ssh -i backups.id_rsa backups@source.example.com
exit
vim nightly-backups.sh

I've based my backups script on the one in the Arch Wiki

#!/bin/bash

# -a Archive mode (keep file permissions etc...)
# -A preserve ACLs
# -X keep extended attributeds

# Backup www.example.com
rsync -aAX \
    -e "ssh -i /home/michael/backups/backups.id_rsa" \
    --rsync-path="sudo rsync" \
    --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found"} \
    --delete \
    backups@www.example.com:/ \
    /home/michael/backups/www.example.com/

And link the script to crontab

# The script needs to be owned by root or it won't execute.
sudo chown root:root nightly-backups.sh
sudo chmod 774 nightly-backups.sh
sudo vim /etc/crontab

# rsync servers back here 01:15 every day.
15   1  *   *   *   root    /home/michael/backups/nightly-backups.sh

The reason I've got the backup job also running as root on the destination server is we are keeping the file permissions and some files will be owned by root (from the source server) so we need to be root on the destination server to overwrite them when they change.

It's not the best backup solution but it's simple and effective.

I'm an extremely satisfied customer and I'd be happy to recommend them. If you are looking for reasonably priced and reliable servers in Oceania with excellent customer service RansomIT are great. ↩
I wouldn't recommend just syncing a server as a backup solution, if you get hit by cryptolocker and you sync your files then your backups are encrypted too. But in this case the synced copy is getting backed up by Crashplan which handles all of the file revisions and retention time frames. ↩
On the Unix and Linux stackexchange Martin von Wittich makes a good point that this could be run as root. He is correct, my choice to use a seperate account is not that it directly increases security but I feel it's neater and it doesn't lead to sprawl.

For example say another job also needs to ssh in as root, if there is already an SSH key it would so easy to use the same key pair rather than generate a new one and append it to the authorized_keys. But then if you want to disable the backups you need to work out what other services are using that key pair.

I've seen Windows environments where twenty or thirty different services were running as the domain administrator account, many of these service did need administrative access, things like backups, inventory systems and anti-virus. But it meant that the domain admin password couldn't be changed (for example when staff left) without breaking things. It took a lot of work to find all of the things that were using the account and migrate them to their own accounts so we could change (and disable) the domain admin account. ↩

Adding subject alternative name to certificate request

2016-10-20T07:00:00+08:00

This is another note to self, I must have done this 20 or 30 times over the years but I can never remember exactly how.

The easiest way I've found to add subject alternate names to certificate requests is to add two lines at the end of /etc/ssl/openssl.cnf

[SAN]
subjectAltName=DNS:example.com,DNS:www.example.com,DNS:mail.example.com

Then when creating a CSR simply include -reqexts SAN

openssl req -out CSR.csr -new -newkey rsa:4096 -nodes -keyout privateKey.key -reqexts SAN

openssl req -out CSR.csr -key my-existing-key.example.com.key -new -sha256 -reqexts SAN

It's that simple.

Recovering data from a hard drive after wiping the partition table

2016-10-13T07:00:00+08:00

A mate from work had a LUKS volume that he had setup and was using to store his personal documents, things like scanned copies of invoices and tax information. Unfortunately in a momentary lapse of concentration after mounting it he ran mkfs.ext4 over it.

I'm sure we have all been there before, anyone who has spent enough time in tech knows the gut wrenching feeling after entering the wrong command. I can still remember the tense feeling after I ran query but forgot the where clause and saw (2986 row(s) affected) when I was expecting (1 row(s) affected).

The good news was that my mate had a backup, the bad news was the backup was six months out of date. He was able to make a copy of the LUKS volume and run PhotoRec over it which pulled back all the files.

PhotoRec is a brilliant tool but it can't recover metadata so things like file names, creation and last modified dates and directory structures were all missing. So he was left with thousands of recovered files, some of which he already had and others that were new.

I wrote a simple python script to run through two directories and check for files in the new directory (the recovered files) that were not in the old directory (backup of original files).

#!/usr/bin/python3
# -*- coding: UTF-8 -*-
"""
A small python script to find new files in two similar directories.
"""

import os
import os.path
import hashlib
import argparse


def setup_options():
    """
    Parse options and get the location of the old and the new directory.
    """
    parser = argparse.ArgumentParser(
        description=('Run through two directories (including sub directories) '
                     'and find files that are the new directory but not in '
                     'the old directory.'))
    parser.add_argument(
        'old_files',
        metavar='old_directory',
        type=str,
        help='The old directory with the original files')
    parser.add_argument(
        'new_files',
        metavar='new_directory',
        type=str,
        help='The new directory with both original files and new ones.')

    return parser.parse_args()


def compare_two_directories(settings):
    """
    Run through two directories (including sub directories) and find files that
    are in the new directory but not in the old directory.
    """
    original_files = set()

    # Run through the original directory and creates an MD5 sum each of the
    # files. MD5 is insecure because of known hash collisions, however we are
    # not trying to validate the file's contents so it's good enough, faster
    # and more memory efficient than SHA256.
    for dirpath, dirnames, filenames in os.walk(settings.old_files):
        for filename in filenames:
            file_path = os.path.join(dirpath, filename)
            file_hash = hashlib.md5(open(file_path, 'rb').read()).hexdigest()
            original_files.add(file_hash)

    # Run through the new directory and print files who's md5 hash is not in
    # the original list of files.
    for dirpath, dirnames, filenames in os.walk(settings.new_files):
        for filename in filenames:
            file_path = os.path.join(dirpath, filename)
            file_hash = hashlib.md5(open(file_path, 'rb').read()).hexdigest()
            if file_hash not in original_files:
                print(file_path)


if __name__ == "__main__":
    compare_two_directories(setup_options())

I've put this script up on GitHub so anyone can use it, with a simple

michael@xo:~$ ./compare.py /home/michael/backup /home/michael/recovered-files

It runs through all the files in the old directory (and it's subdirectories) and calculates an MD5 sum. While MD5 is broken for validating the contents files because of known hash collisions, and should never be used for storing passwords, we are just trying to compare two documents neither of which is coming from an untrusted source so it's good enough and quicker when running over a few thousands documents than SHA256¹.

Then it stores the MD5 hash in a set. I've used a set rather than say a list, because I don't want duplicates and I want to be able to check if a value is in the set quickly.

Next it runs through the new directory (and it's subdirectories) and for every file that has an MD5 sum that's not in the set, it outputs the name.

This script brought the number of recovered files down from thousands to a manageable amount, so hopefully it's useful for someone else in a similar situation.

I believe that thanks to optimizations in the design of SHA256 it can potentially be quicker than MD5. I've heard that with OpenSSL SHA256 is quicker than MD5. But with my tests using python's hashlib MD5 was faster than SHA256. ↩

Setting up mailpile on Debian 8 Jessie for remote access

2016-10-06T07:00:00+08:00

I've never yet found a mail client that I really love. Outlook is ok in corporate environments. Gmail is probably the best I've used but I'm trying to move away from Google and free 3rd party hosted solutions. Roundcube is pretty good, I'm not a fan of PHP but most of the internet seems to run on it. I've run Rainloop for a while I like it. I actually think KMail and Kontact as a whole are very nice. I kind of wish they handled HTML email a bit more elegantly but with some tweaking they work well. The problem is they are desktop solutions and I'd like something I can access remotely. I use Mutt over ssh a bit and that works well enough.

So I though I'd give mailpile a go, I know it's not designed to run as a web service but it can.

The install guide is very well documented. I pretty much followed it verbatim although I tweaked a few things because I wanted it to run as a service.

I SSHed into my home box and forwareded port 33411

ssh example.com -L 33411:localhost:33411

Install the prerequisites

sudo apt-get install git gnupg openssl python-virtualenv python-pip python-lxml python-dev libjpeg-dev

Clone the Git Repo

 sudo git clone --recursive https://github.com/mailpile/Mailpile.git /opt/mailpile

Create a mailpile user

For now we are going to give the user /bin/bash as a shell, but later we will change it to /usr/sbin/nologin

sudo useradd --system --shell /bin/bash --home-dir /opt/mailpile mailpile

Setup the virtual environment

Unfortunately it looks like mailpile only runs with Python 2.7 not Python 3+

# Set ownership of mailpile
sudo chown -R mailpile:mailpile /opt/mailpile/

# Change to the mailpile user
sudo su mailpile

# move into the newly created source repo
cd /opt/mailpile

# create a virtual environment directory
virtualenv -p /usr/bin/python2.7 --system-site-packages mp-virtualenv

# activate the virtual Python environment
source mp-virtualenv/bin/activate

Install the dependencies

pip install -r requirements.txt

Run mailpile

As a test we are going to start mailpile manually, once we are sure it's working we will make it a service.

./mp

If all has gone well you should now see a mailpile> prompt and if have forwareded the ports you should be able to browse to http://localhost:33411

Setup mail

The setup was very simple,

You are first greeted with a welcome screen where you select your language

Next you choose a password

Click through to finish the setup

And you end up a login screen

Once you login you are presented with a welcome screen.

Before you can add an account you need to run through the privacy settings, I went with the defaults which were pertty good.

Then you go back to the welcome screen but this time you can add an account.

I added an account

I went with the 'Detect settings' option to see how well that worked, it took a couple of minutes but got everything right. Later I also tried adding settings manually and that was pretty easy too.

Lastly you setup your encryption options and your done.

Now you can check your mail.

When I was setting up Mailpile I ran into issue 1578 and so no mail was showing up. Fortunatly there is a simple fix for that which will hopefully be merged soon.

I'm planning on running Mailpile under a subdirectory (e.g. example.com/mailpile) so in the mailpile terminal I ran set sys.http_path = /mailpile but that's not necessary if your planning on running it in the root of your domain.

mailpile> login

Your password:

mailpile> set sys.http_path = /mailpile
Elapsed: 0.001s (set: Updated your settings)

{
    "sys.http_path": "/mailpile"
}

mailpile>

Make it a service

Press Ctrl + D to exit the mailpile cli, then type deactivate python virtual environment and exit to change back to your normal account.

sudo vim /etc/systemd/system/mailpile.service

[Unit]
Description=Mailplie Server
After=syslog.target
After=network.target

[Service]
Type=simple
User=mailpile
Group=mailpile
WorkingDirectory=/opt/mailpile
ExecStart=/opt/mailpile/mp-virtualenv/bin/python mp --www= --wait

# Give a reasonable amount of time for the server to start up/shut down
TimeoutSec=300

[Install]
WantedBy=multi-user.target

Enable and start the service

sudo systemctl enable mailpile.service
sudo systemctl start mailpile.service

Now that it's a service we can lock the user account down a bit more by giving it no shell.

sudo usermod -s /usr/sbin/nologin mailpile

Make it a website

As Mailpile is still in beta and they recommend you don't leave it open.

At the moment, we do not recommend exposing Mailpile directly to the wider Internet.

So I'm going to be setting mine up so you need a client certificate to access it. This isn't necessary but it's an extra layer of security and I'd recommend it. I've got a tutorial on setting up a personal certification authority and securing apache with client certificates if you want more information on how that works.

sudo vim /etc/apache2/sites-enabled/000-default.conf

Below is most of my apache2 config, but the important bits are under # SSL Client Certificates and # Mailpile

<VirtualHost *:443>
    # Host settings
    ServerName example.com

    # SSL Settings
    SSLEngine on
    SSLOptions +StrictRequire
    SSLHonorCipherOrder on

    # Remove all, Add back only TLS1.2
    SSLProtocol -ALL +TLSv1.2

    # A fine selection of the choicest ciphers
    SSLCipherSuite -ALL:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256

    SSLCertificateFile /etc/ssl/custom/example.com.crt
    SSLCertificateChainFile /etc/ssl/custom/1_intermediate.crt
    SSLCertificateKeyFile /etc/ssl/private/exmple.com.key

    Header always add Strict-Transport-Security "max-age=31536000"   

    # HPKP: HTTP Public Key Pinning
    # https://scotthelme.co.uk/hpkp-http-public-key-pinning/
    # https://developer.mozilla.org/en-US/docs/Web/Security/Public_Key_Pinning
    Header always set Public-Key-Pins "pin-sha256=\"f5uthPZ21VOlA6Bye2yvoe+6a/h9fKRK27SdFt43XHQ=\"; pin-sha256=\"ATwpV5xzLfkVs631iympx7q+JlvRePMgTcvFG7x3Eeo=\"; max-age=5184000; includeSubDomains"


    ServerAdmin webmaster@example.com
    DocumentRoot /var/www/html

    # Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
    # error, crit, alert, emerg.
    # It is also possible to configure the loglevel for particular
    # modules, e.g.
    #LogLevel info ssl:warn

    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined

    ################################################################################    
    # SSL Client Certificates
    ################################################################################        

    # This allows any client certificates issued by Example CA
    SSLCACertificateFile /etc/ssl/custom/ExampleCertificationAuthority.pem

    ################################################################################    
    # Mailpile
    ################################################################################    

    ProxyPass /mailpile http://127.0.0.1:33411/mailpile
    ProxyPassReverse /mailpile http://127.0.0.1:33411/mailpile

    <Location /mailpile>
        SSLVerifyClient require
        SSLVerifyDepth 3
        # Restricts the list of client certificates we accept, from all
        # client certificates issued by Example CA to just authorised ones.
        SSLRequire %{SSL_CLIENT_S_DN_Email} eq "michael@example.com" \
                or %{SSL_CLIENT_S_DN_Email} eq "michael@xo.tc"
    </Location>
</VirtualHost>

Restart Apache and your done.

sudo systemctl restart apache2.service

My ownCloud update script

2016-09-29T07:00:00+08:00

Update 2017-03-02: If your using Nextcloud they now have their own in browser updater this script still works though.

One of the most important things in security is patching. For the last several years two out of the ASD's top four mitigations have been patching (patching applications and patching the Operating System).

To me I think how your going to get new version of your application out to your end users should be a decision made very early in the design phase. About the time your thinking whats the most appropriate programing language to tackle a problem you should also be thinking how will I deploy this code once it's written and how will we update deployments.

Sometimes this decision will be made for you by the platform you are targeting such as Android or IOS. Sometimes it's not your problem when you expect downstream distributions to deal with packaging and updates¹. And sometimes you need to build your own update mechanism. I quite like the way LibreNMS updates, which basically boils down to a cron job doing a git pull every day.

I've used ownCloud for a while but one thing that's always annoyed me is they don't have an easy way to upgrades. I'd like to just tick a box that says 'Keep me on the latest stable version' or at the very least 'download security fixes automatically'.

For now I've made do with this script and just manually downloading the latest version of ownCloud each time there is a patch.

I've created a directory called /opt/owncloud-install/ and in there I've got a subdirectory called old-installs. When a new ownCloud version comes out I cd into /opt/owncloud-install/ wget the latest version and run /opt/owncloud-install/upgrade.sh

#!/bin/bash

# Stop Apache2 (not necessary, but it's a good idea)
systemctl stop apache2.service

# Backup config.php and data
mv /var/www/owncloud/config/config.php /opt/owncloud-install/config.php
mv /var/www/owncloud/data /opt/owncloud-install/

# Delete everything else
rm -rf /var/www/owncloud/
# Extract a fresh copy (the tar ball doesn't include a data and config.php)
tar -xf owncloud-*.tar.bz2 -C /var/www/

# Replace the backed up files
mv /opt/owncloud-install/config.php /var/www/owncloud/config/
mv /opt/owncloud-install/data/ /var/www/owncloud/

# Set ownership (the files in the tar ball are owned by nobody)
chown -R www-data:www-data /var/www/owncloud/

# Start Apache2 back up.
systemctl start apache2.service

# Achive install file
mv owncloud-*.tar.bz2 old-installs/

After that you still need to visit your ownCloud home page and click through the database upgrades.

It's not the most robust script but it works well enough for me. I've been looking at NextCloud, I haven't made the switch yet but if they introduce an automatic update mechanism that would be a big enough draw card for me to change.

Although even then you need some way to mark a new version and security fixes so the downstream can package them. ↩

Calls from the bank

2016-09-22T07:00:00+08:00

I bank with one of the big four banks in Australia I recently got a call, and it started off

"Hi I'm Taylor¹ I'm calling from {bank_name} and I'd like to talk to you about a letter you received from the bank recently"

I hadn't seen any letter but I've migrated all my bills to paperless and I on only check my snail mail once every week or two. I said so and the call continued.

"I apologizes for this but I need to let you know this call is being recorded for quality assurance purposes, and as I am accessing your profile I will need to verify your account, I'll need your full name including middle name, current address and date of birth."

At this point Taylor had provided me with no details to prove the call has come from the bank. The call had come from an odd number² and they were asking for personal information.

So I asked Taylor if I could call back to verify the call was from the bank and also asked for Taylor's last name. Taylor seemed genuinely surprised and almost a little offended that I didn't just trust that this mysterious call was from my bank. Taylor then told me that employees were not allowed to give out their last name for privacy reasons but suggested that I call back on the same number that I had in my caller ID. That number is not publicly available anywhere on the bank website. In fact when I Googled the number I found other people asking about the same number on the bank's forums with a moderator saying "I am unable to confirm whether this is a {bank_name} phone number;"

So I called the bank using the number on their contact us page and I was expecting to hear them say, "Yes it's a scam, it's been doing the rounds and we are doing our best to stop it but comes from outside Australia and it's hard to shutdown." Instead I spoke to Sandy³ who actually laughed and also seemed very surprised that I'd though someone cold calling me and calming to be from a bank with about a 20% market share⁴ might not be genuine.

As it turned out the call was genuine and they were trying to sell me a an "upgrade" on my mortgage.

I saw Tory Hunt write about this almost two years ago. I thought banks were getting on top of this sort of thing, and I thought my bank had a pretty good security team. I met a few people who claimed to be part of their security teem at Ruxcon and they seemed pretty switched on.

I guess this is the sort of thing a marketing manager might setup without really thinking about the security implications but it's almost training people to just accept cold calls and give out information. I'm amazed they don't get more people calling back to verify but it seemed like I was an anomaly.

I've changed the name. ↩
I'm aware that caller ID spoofing is not that hard but it still, it was a number I didn't recognize. ↩
Again I've changed the name. ↩
I've looked for a reliable source on banks market share and they all seem to differ a bit but it's generally about 20% to each of the big for banks and the remaining 20% split between all the small credit unions and little one branch banks. That combined with the fact that many people use more than one bank, I think if you called random Australian phone numbers you would have better than a 1 in 5 chance of finding someone who uses this bank. ↩

An excellent demonstration of Cross-Site Request Forgery

2016-09-15T07:00:00+08:00

A few weeks ago one of the people I follow on GitHub stared the superlogout repository.

Intrigued by the name I went to check it out and found a simple site that logs you out of a bunch of services. For those that want to try it, it's available at superlogout.github.io the way it works is pretty straight forward, it uses JavaScript to GET a bunch of urls that are the logout pages for services (or in the case of YouTube, DeviantART and LiveJournal it's a POST).

This an excellent demonstration of how Cross-Site Request Forgery works. In this case it's made clear that they are logging you out of the service, but they don't have to show you. I could embed the same JavaScript in my site but not show anything, people would just be mysteriously logged out of their GMail after reading my blog. Further I could use that along with my analytics to record which services visitors were logged into when they visited.

Logging someone out is not that dangerous but it's easy to see what could be done without CSRF protection. If they could post to any page on those sites with JavaScript, they could buy things on Amazon, bid on auctions on eBay, send emails with GMail and so on.

It's not entirely clear to me whether this is a parody site showing the power of CSRF or if it's genuinely meant to be a service. But my feeling is that it's meant to poke fun at the insecurities, and demonstrate that CSRF protection is needed on all pages, including logout pages and not just on pages where you can post data.

Alternative Networks for this site - ZeroNet

2016-09-08T07:00:00+08:00

Over the last couple of weeks I've published this site as an I2P eepsite, and as a Tor Hidden Service. This week I'm announcing Exotic Security is now available as a ZeroNet site.

http://localhost:43110/19M77j42ddq7wgvZctRSxR8Dyq7De8SGYb

I think ZeroNet is the most conceptually different network I've looked at yet. While all the other networks I've looked at have been the standard client server model, I think of ZeroNet more like a torrent file that contains a bunch of HTML pages. So you can get the files off a peer to peer network and when you view the site you're both the client and the server.

ZeroNet is designed to be censorship resistant in the same way that torrents are, but also like torrents it's not designed for privacy.

As far as I can tell there is no official way of running ZeroNet as a service. I understand that it's designed to be peer to peer so your site can still be served even if your host is offline, but I wanted to run it as a service so I know there is always at least one host seeding the latest version of my site.

These are my notes on installing ZeroNet on Debian Jessie. All commands run as root (with sudo).

apt-get install python-msgpack python-gevent
git clone https://github.com/HelloZeroNet/ZeroNet.git /opt/zeronet
useradd --system --shell /usr/sbin/nologin --home-dir /opt/zeronet zeronet
chown -R zeronet:zeronet /opt/zeronet/
vim /etc/systemd/system/zeronet.service

zeronet.service copied from Bruno Loff's Ubuntu Install

[Unit]
Description=Zeronet Server
After=syslog.target
After=network.target

[Service]
Type=simple
User=zeronet
Group=zeronet
WorkingDirectory=/opt/zeronet
ExecStart=/usr/bin/python zeronet.py

# Give a reasonable amount of time for the server to start up/shut down
TimeoutSec=300

[Install]
WantedBy=multi-user.target

systemctl start zeronet.service
systemctl enable zeronet.service

Getting a ZeroNet site up and running was pretty easy but I ran into some issues with my pelican site because I've been using root relative URLs but in ZeroNet my site is not running in the root of the server but instead under /19M77j42ddq7wgvZctRSxR8Dyq7De8SGYb/ so I had to go back and look at the documentation and update all the internal links in all my posts. After about half an hour with find and replace, grep and some regex I was in back in business.

Alternative Networks for this site - Tor

2016-09-01T07:00:00+08:00

Last week I looked a few alternative censorship resistant networks and setup an I2P eepsite. This week I've made Exotic Security available as a Tor hidden service.

http://exoticsecv6kd6fw.onion

I like vanity domain names so first I downloaded Scallion and generated an onion address that started with 'exoticsec'.

Scallion was very easy to use, just a simple git clone, xbuild and then

mono scallion.exe -c exoticsec

I was very impressed with how well it ran, my GPU a GeForce GTX 680 got 470 MH/s and found two names that matched in under 10 hours¹.

I then installed Tor following their guide for Debian and set it to run automatically sudo systemctl enable tor.service

The I setup apache, I edited /etc/apache2/ports.conf

# Tor Hidden service
# Just a random port number I generated, there is no significance to it.
Listen 127.0.0.1:9625

my sites-enabled

# Tor Hidden Service
<VirtualHost 127.0.0.1:9625>
    # Host settings
    ServerName exoticsecv6kd6fw.onion

    ServerAdmin webmaster@xo.tc
    DocumentRoot /var/www/tor-hidden-service
    ErrorDocument 404 /pages/404-not-found.html

    # Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
    # error, crit, alert, emerg.
    # It is also possible to configure the loglevel for particular
    # modules, e.g.
    #LogLevel info ssl:warn

    # As we are hosting on localhost, by default the server-status and
    # server-info pages are avalible.
    <Location /server-status>
        Order allow,deny
        Deny from all
    </Location>
    <Location /server-info>
        Order allow,deny
        Deny from all
    </Location>


    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

Then I edited /etc/tor/torrc and uncommented the two lines to enable a hidden service

HiddenServiceDir /var/lib/tor/hidden_service/
HiddenServicePort 80 127.0.0.1:9625

Restarted tor to create the HiddenServiceDir

systemctl restart tor.service

Then I replaced /var/lib/tor/hidden_service/private_key with the key I'd generate with Scallion and I was done.

I was surprised how easy it was to get up and running. Admittedly it might have been a bit more involved if I'd been trying to hide my identity, I probably would have used Whonix as the host instead but even so it was very easy to get up and going.

Although I'm pretty sure that was mostly luck, the predicted time for one hash that matched was a little over 10 hours. ↩

Alternative Networks for this site - I2P

2016-08-25T07:00:00+08:00

I've been very interested in all the different censorship resistant that seem to have sprung up over the last few years so I thought I'd have a look at hosting this site on them. It seems like the ideal site to try them out with because I use pelican to generate static html files and they should be fairly easy to host anywhere.

The some of the networks I've looked at recently are:

and this week I've started running Exotic Security as an I2P eepsite¹ it's now available at:

http://xotc.i2p

http://gqgvzum3xdgtaahkjfw3layb33vjrucmw5btyhrppm463cz3c5oq.b32.i2p/

I've used I2P for a while now and it's fairly similar to its more popular cousin Tor although there are a few notable diffrences. The ones that stand out for me are:

I2P was designed was a global passive adversary in mind. Someone who can watch the whole network, every packet that goes in and every packet that comes out. So it uses tricks like constantly sending some amount of traffic, whether you're using it or not to thwart traffic flow analysis.
Unlike Tor who ask you not to torrent over their network I2P actually encourages torrents and has a a built in torrent engine called snark.

Setting up an eepsite was fairly easy. I installed I2P simply following their debian install guide.

I decided to go with a site run by Apache and use I2P as a reverse proxy rather than use the built in web server.

I edited the ports that apache listens on vim /etc/apache2/ports.conf

# I2P eepsite
Listen 127.0.0.1:7658

and added an entry in /etc/apache2/sites-enabled/000-default.conf

<VirtualHost 127.0.0.1:7658>
    # Host settings
    ServerName xotc.i2p

    ServerAdmin webmaster@xo.tc
    DocumentRoot /var/www/eepsite
    ErrorDocument 404 /pages/404-not-found.html

    # Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
    # error, crit, alert, emerg.
    # It is also possible to configure the loglevel for particular
    # modules, e.g.
    #LogLevel info ssl:warn

    # As we are hosting on localhost, by default the server-status and
    # server-info pages are avalible.
    <Location /server-status>
        Order allow,deny
        Deny from all
    </Location>
    <Location /server-info>
        Order allow,deny
        Deny from all
    </Location>

    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

and in the router console under I2P internals > Hidden Service Manager I enabled the website.

I could have simply pointed I2P at the site already running on port 443 but Pelican uses absolute links, also I use Piwik analytics which again points to a clear net site. So instead I've used pelican to generate a new site with the relative links and no analytics.

I have a feeling I saw somewhere that I2P were looking at moving away from the name 'eepsite' and moving to calling them 'Hidden Services' like Tor does, but now I can't find that anywhere so I'm sticking with eepsite. ↩

Fingerprint readers on phones

2016-08-18T07:00:00+08:00

I can clearly remember my first reaction when I heard about fingerprint readers on phones. It was a stream of thoughts along the lines of "I bet that will be broken in an matter of days, biometrics are not ready for prime time", "You don't go around leaving a smudgy copy of your password on every glass you hold" and "You can change your password but good luck resetting your fingerprint after that gets compromised."

But I was falling into a trap that's all to common in information security¹ of rejecting an idea because it isn't perfect. When the question that I should have been asking isn't "Is it flawless?" but "Is it better than what we currently have?".

Shortly after my initial reaction I started thinking a little more deeply about the idea and I could think of a number of friends and family that didn't even use a pin on their phone because it was too much effort to unlock every time. I decided that if a fingerprint reader was significantly more convenient and if that was enough to get people to lock their phone then it would be a net win for security.

I recently bought a Nexus 6P and installed CyanogenMod. Now that I've got a fingerprint reader I think it's just brilliant. Previously I used a pattern to lock my screen and for my encryption key because it was quick and easy. Now I use a 16 character password² which is hard to type for the lock screen and encryption key. Then when I want to unlock it for every day use I just use my fingerprint.

I still need to enter my password to decrypt my phone if I reboot it. And every three days it times out but because I don't have to enter it every single time I unlock it, it's not too much of a hassle so I don't mind having a longer and more secure password.

It's not just an issue in InfoSec, comes up in all areas of life. ↩
I'd like to use a longer passphrase but unfortunately 16 character is the limit for now. ↩

How often should you change your password?

2016-08-11T07:00:00+08:00

There was an article recently based on a 2010 study that suggested that frequent password changes actually negatively impact security.

I agree with the article but feel that some of the commentary could be a little more nuanced. The thrust of the article is that forcing regular password changes irritates users and they turn to patterns like adding a number on the end and just incrementing it. I've seen my share of users who will happily announce to the world that their password is 'August2016' without being asked.

If MySpace forced passwords to expire every 30 days and you found 'March2008' in the recent MySpace breach, it wouldn't take a genius to work out what was next.

On the other hand, occasional password changes do have their place. Should MySpace wait until there is public evidence of a breach to force a reset? What about other organisations like Facebook and Netflix who reset passwords of users because they had reused passwords their MySpace password? There are a lot of ifs here;

If MySpace had used a secure hash like PBKDF2 or bcrypt then all but the weakest of passwords would be secure and;
If users picked passwords with a significant amount of entropy they wouldn't be cracked even if MySpace just used md5 and;
If users didn't reuse passwords across sites it wouldn't be an issue for other site anyway and;
If ...

Our natural response as security professionals is to try to force people to use good password hygiene so we resort to things like password expiry dates and complexity requirements that actually restrict the number of available passwords.

We should look at ways to encourage good password hygiene, sign up forms can offer the option to generate a password¹ and a button to copy it into the clipboard so users can use a password manager. Maybe organisations should include tools like KeePass as part of the standard operating environment and make them available on all desktops.

In the future we need to move to things like two factor authentication with a hardware device and reduce our reliance on passwords.

My preferred method of generating passwords is dd if=/dev/random bs=1 count=18 2>/dev/null | base64. I haven't looked into this at all, but I'm sure there would be a secure way to do the equivalent of that in client side JavaScript. ↩

Kerckhoffs's principle

2016-08-04T07:00:00+08:00

One of the security tenants that I live by is Kerckhoffs's principle.

A cryptosystem should be secure even if everything about the system, except the key, is public knowledge.

-- Auguste Kerckhoffs, 1883

It's beautiful in it's simplicity and yet counter intuitive.

It's a beguiling myth that if you want to make a system secure you should make it secret. Hundreds of years of experience have thought us that for a system to be truly robust it needs to be open and auditable.

Kerckhoffs's principle is applicable to so much more than just cryptography. I think it needs to be much broader, and apply to any system designed to provide security.

A security system should be secure even if everything about the system, except the key, is public knowledge.

I recently spent a little time looking at physical security controls, things like security cameras and digital locks (RFID cards). It makes my blood boil when I see how much vendors try to restrict information and refuse to publish even a basic manual.

Especially as often once someone takes a good look their products turn out to be riddled with vulnerabilities. Then rather than fix the vulnerabilities vendors try to use things like the Digital Millennium Copyright Act (DMCA) to silence security researchers and prevent the information spreading.

Updating UEFI BIOS on Lenovo ThinkPad X220

2016-07-28T07:00:00+08:00

I've got a ThinkPad X220 and I've been a bit lax about patching the UEFI/BIOS. But recently this SMM "Incursion" Attack has been getting a bit of publicity and it's spurred me to try to patch it. It's worth noting that this bug is not specific to Lenovo, but something that affects most vendors of Intel based hardware. For those interested some of the best coverage I've found was from a recent risky business podcast¹.

Unfortunately Lenovo don't provide a way to upgrade the BIOS from Linux. While the ThinkPad X220 is officially supported with Linux (RedHat and Fedora) the only BIOS update utility they provide is for Windows.

I went on to the Lenovo support site and for the BIOS update, under Operating system I picked "Not Applicable" a few patches BIOS and Firmware patches came up but all of them were .exe files.

After a long time of searching I found that they do provide a .iso file of a bootable CD to patch your BIOS but you have to pick Windows as the operating system to find it. I have no idea why they think that .exe files are "Not Applicable" but an iso is a Windows specific option but maybe they didn't put much thought into it.

Now that I had an iso I was half way there, but the ThinkPad X series don't have an optical drive and simply dd'ing the image onto a flash drive didn't work.

I found some instructions on ThinkWiki with a link to a perl script that could create a bootable image.

I copied the image onto a flash drive but when I tried to boot it failed, after a bit of experimentation I found that the instructions on ThinkWiki were not quite right, they recommended setting the boot to UEFI only but I found that I needed to use Legacy BIOS to boot the flash drive.

After that I was able to boot the drive and update my BIOS.

Final steps

So in summary the steps that finally worked were:

Download the iso file.
Convert the iso file to a bootable image.
1. wget https://userpages.uni-koblenz.de/~krienke/ftp/noarch/geteltorito/geteltorito.pl
2. perl geteltorito.pl -o ThinkPad-x220-bios-update.img 8duj27us.iso
Copy the image onto a flash drive
1. Check the name of your flash drive first! dd if=ThinkPad-x220-bios-update.img of=/dev/sdb
Set boot to "Both" Legacy and UEFI
1. Reboot, pressing F1 while booting to enter setup
2. Go to Setup > UEFI/Legacy Boot > Both
3. F10 to save and exit
Boot the flash drive
1. Press F12 while booting to and select your flash drive.

Follow the prompts to upgrade your UEFI/BIOS and your done.

Risky Business #417 from 14:20 to 18:00 ↩

Setting Up Full Disk Encryption on Arch Linux

2016-07-21T07:00:00+08:00

I recently switched my laptop over from Debian Stretch to Arch Linux.

Debian is still my go to distribution for any server, but I felt like I was in a bit of a no man's land with my laptop. Debian stable (currently Jessie) is rock solid and reliable but I want to install new packages, like the latest version of Firefox. Debian testing and unstable (Stretch and Sid) are well ... unstable, and you really can't complain when things break¹.

I've been using Arch Linux on my desktop for a while and for a bleeding edge distribution it's surprisingly stable².

These my notes on installing Arch Linux on my laptop with Full Disk Encryption. As I noted in my post on Setting Up Full Disk Encryption on Debian Jessie it's not really "Full" disk encryption, there is still a small partition /boot that's unencrypted and will contain the kernel and initramfs.

Download

First I downloaded the latest Arch Linux iso, verified it, and copied it to a USB flash drive.

gpg --recv-keys 0x7f2d434b9741e8ac
gpg --verify archlinux-2016.07.01-dual.iso.sig
dd if=archlinux-2016.07.01-dual.iso of=/dev/sdb

Install

Then I booted the USB drive and, updated the time, because that's what it says in the documentation and it's a good idea.

timedatectl set-ntp true

I created 3 partitions on my hard drive an 80GB SSD:

512MB FAT32 partition to boot from.
A big partition to be used as the root.
A 4GB partition for swap space.

parted /dev/sda
 > mklabel gpt
 > mkpart ESP fat32 1MiB 513MiB
 > set 1 boot on
 > mkpart primary 513MiB -4G
 > mkpart primary 76GB 100%

I setup a LUKs volume on the second partition, formated it to btrfs then mounted it to /mnt/ and mounted the FAT32 volume to /mnt/boot

cryptsetup luksFormat /dev/sda2
cryptsetup luksOpen /dev/sda2 arch_root

mkfs.btrfs /dev/mapper/arch_root

mount /dev/mapper/arch_root /mtn/
mkdir /mnt/boot
mount /dev/sda1 /mnt/boot

I updated the mirrors list and installed Arch Linux following the install guide and installed vim³ and added a host name.

vim /etc/pacman.d/mirrorlist

pacstrap /mnt base

genfstab -p /mnt >> /mnt/etc/fstab

arch-chroot /mnt

ln -s /usr/share/zoneinfo/Australia/Perth /etc/localtime

nano /etc/locale.gen

locale-gen

nano /etc/locale.conf
# LANG=en_AU.UTF-8

pacman -S vim

vim /etc/hostname

I edited /etc/mkinitcpio.conf to add encrypt after udev on line 51 and then created my initramfs

vim /etc/mkinitcpio.conf
# HOOKS="base udev encrypt autodetect modconf block filesystems keyboard fsck"
mkinitcpio -p linux

Next I used efibootmgr to add an entry into my EFI boot options to boot the Linux kernel directly rather than using a boot loader like GRUB which then boots the kernel.

I think this is an extremely elegant solution as it means I end up with only 3 files in my /boot/ volume: The kernel, The initramfs, and a fallback initramfs (which isn't really necessary). It's much neater than a bunch of GRUB scripts and config files.

pacman -S efibootmgr

efibootmgr -d /dev/sda -p 1 -c -L "Arch Linux" -l /vmlinuz-linux -u "cryptdevice=/dev/sda2:arch_root root=/dev/mapper/archroot rw initrd=/initramfs-linux.img"

Next I added a user and setup sudo so they could become root.

useradd michael --create-home --groups wheel
passwd michael

pacman -S sudo
visudo
# Uncomment line 82. %wheel ALL=(ALL) ALL

Lastly I setup my swap partition to be a LUKs volume with a random key.

vim /etc/crypttab
# arch_swap /dev/sda3 /dev/urandom swap

vim /etc/fstab
# /dev/mapper/arch_swap none swap sw 0 0

I exited the chroot and rebooted.

exit
reboot

Post install

I installed KDE because that's my desktop of choice but the lovely thing about Arch Linux is you can make it almost anything you want.

# Logged in as Michael, but run as root
dhcpcd enp0s25

pacman -S xorg-server
pacman -S plasma-meta
pacman -S kde-applications-meta
pacann -S sddm

systemctl enable sddm.service
systemctl enable NetworkManager.service

Added the track pad driver

pacman -S xf86-input-synaptics

Set time to NTP, I would have though this would be done automatically because I'd synced the time when I started the installer, but apparently not.

timedatectl set-ntp true

And set the KDE Wallet to automatically unlock with my user password. This is a slight trade off in security because with the default setup I could have two different passwords, or I could login but choose not to unlock the wallet. But in this case I've decided to go with it because it's much more convenient and secure enough.

sudo pacman -S kwallet-pam

vim /etc/pam.d/sddm

My sddm file

#%PAM-1.0

auth            include         system-login
auth            optional        pam_kwallet5.so
auth            optional        pam_kwallet.so kdehome=.kde4

account         include         system-login

password        include         system-login

session         include         system-login
session         optional        pam_kwallet5.so
session         optional        pam_kwallet.so

You can't complain but you can file bug reports, which is helpful to the Debian maintainers. ↩
Things still break in new and interesting ways on Arch Linux, just less often than I would expect for the rate of package churn. ↩
Vim is included in the installer .iso file, so you can use it while your installing, but it's not part of the base packages so once you run arch-chroot you can't use it until you install it pacman -S vim ↩

StartSSL launches StartEncrypt

2016-07-14T07:00:00+08:00

Let's Encrypt has been shaking things up in the Certification Authority world. Let's Encrypt certificates are free, automated and easy to install. They have been gaining market share like crazy. Some CA's have reacted to their loss of market share in interesting ways.

Let's Encrypt are not perfict, but they are good and they are available now rather than spending another 6 years in development trying to achieve perfection.

They have some notable (and largely intentional) limitations:

They don't do Extended Validation certificates.
They don't do wildcard certificates.
They don't issue certificates for internal servers can't be accessed from the internet¹.
They don't issue client certificates to be used for things like S/MIME.
Certificates are limited to 90 days.

These limitations mean that Let's encrypt is only useful about 99% of the time².

One thing Let's Encrypt was meant to do was make other Certification Authorities innovate, and StartCom have done that. In my opinion they were already one of the innovators in the field. They were giving away free domain validated SSL Certificates and for Extended Validation you could validate once and then get an unlimited number EV of certificates. In other words they were only charging you for things that were not automated which in itself was pretty revolutionary. Their validation process was pretty rigorous and while parts of their UI felt a little clunky it all worked pretty well.

Now they have released StartEncrypt which is clearly designed to go head to head with Let's Encrypt, from their announcement email:

Compare with Let’s Encrypt, StartEncrypt support Windows and Linux server for most popular web server software, and have many incomparable advantages as:

(1) Not just get the SSL certificate automatically, but install it automatically;

(2) Not just Encrypted, but also identity validated to display EV Green Bar and OV organization name in the certificate;

(3) Not just 90 days period certificate, but up to 39 months, more than 1180 days;

(4) Not just low assurance DV SSL certificate, but also high assurance OV SSL certificate and green bar EV SSL certificate;

(5) Not just for one domain, but up to 120 domains with wildcard support;

(6) All OV SSL certificate and EV SSL certificate are free, just make sure your StartSSL account is verified as Class 3 or Class 4 identity.

I don't think their points are worded particularly well; The first point implies that Let's Encrypt can't install certificates, but Let's Encrypt can automatically install certificates for Apache. The second and fourth points are basically the same. And the fifth point implies that Let's Encrypt can't handle multiple domain names, but it can have up to 100 domain names per certificate, although as said above it won't do wildcards.

Unfortunately the StartEncrypt client appears to be a closed source binary which is a serious problem for a lot of people. I'll admit that I've not read more than a few hundred lines of the Certbot's source code but it's a huge comfort to know that I can if I want to. Also the documentation is fairly thin on the ground, if you download the install file there is an Operating Manual in the doc directory but it's not especially detailed.

If running a closed source binary is not your thing they also have an API, unfortunately at the time of this writing to access the documentation for the API you need to be signed in. From a quick reading it looks to be a fairly simple REST API that you could use to write your own client.

Even though I'll probably be sticking with Let's Encrypt for most things, I think it's great to see some competition.

Also I know StartCom have copped some flack in the past because they gave out free certificates but changed to $25 revoke them. But I think their pricing is reasonable, they charge for manual processes, revocation was a manual process when heartbleed happened so they charged for it. Similarly they offer unlimited free Extended Validation certificates after you have been validated. Validation costs $199 USD. Some people complain that validation isn't free and so EV certificates should not be advertised as free which is fair, but they are fairly upfront about that. And validation it's a real human process, they look at scanned copies of your passport, they call you up on the phone, they require proof that you represent the organization you say that you do none of that is automated.

Another poignant comic by the folks over at commitStrip

I know there are a number of ways you can get a certificate for an internal server, but the design of Let's encrypt is clearly aimed at servers they can directly validate with ACME. ↩
I don't have a source for that, in fact I just made it up. It might be a fun project for someone to run through certificate transparency logs like crt.sh and find out what percentage of certificates issued are just standard dv certs. If anyone does that please let me know. ↩
On their announcement email they directly compare StartEncrypt with Let’s Encrypt ↩