Archive for category Unix / Linux
Replacing Django’s Nasty “runserver”
Posted by Greg in Code, Unix / Linux on December 6, 2011
Have you ever tried to have more than one person view a development site using Django’s built-in development server ? Yeah, it really sucks. Apparently concurrency wasn’t high on the features list and they have stated that it never will be.
DO NOT USE THIS SERVER IN A PRODUCTION SETTING. It has not gone through security audits or performance tests. (And that’s how it’s gonna stay. We’re in the business of making Web frameworks, not Web servers, so improving this server to be able to handle a production environment is outside the scope of Django.)
So how do we go about using something a little nicer without losing any of the auto-reload goodness and without having to setup a full blown production environment ?
There are a number of alternatives, however I’ve selected Twisted Web simply because I really like the twisted framework and due to the experience I have in using it, I am very comfortable with it. It’s a great feature-packed web server that handles concurrency (and a ton of other things) exceptionally well.
So how do we use it to serve our little Django project in a development friendly way ?
I’ve put together some code (some borrowed from other sources) and constructed a simple replacement command called “trunserver” (twisted-runserver). You can grab this code from Github. Simply install it using the standard methods, and run it with:
python manage.py trunserver [IP:PORT] [--settings=foo] [--noreload]
.
So this will start up a twisted web instance serving your Django project and just like the build-in runserver, it will automatically reload your code when it notices that your files have been modified unless –noreload has been passed.
There are a few things missing at this point, like IPv6 support and static file serving, however these are on the roadmap.
I’ll post again with some more info once it is a little more stable and an official release has been provided.
Puppet Modules – Debsecan
Posted by Greg in Unix / Linux on January 2, 2011
This is the first post of (hopefully) many, detailing some of my Puppet module implementations. Being the first, I thought I would start off with something simple.
Debsecan
The debsecan program evaluates the security status of a host running the Debian operation system. It reports missing security updates and known vulnerabilities in the programs which are installed on the host.
This is a great package that I wanted installed on all Debian machines across my entire infrastructure. Thanks to Puppet, this is a breeze.
Module layout
greg@codemine:~/code/puppet %> find modules/debsecan modules/debsecan modules/debsecan/files modules/debsecan/files/debsecan modules/debsecan/files/debsecan-cron modules/debsecan/manifests modules/debsecan/manifests/init.pp
Manifest – init.pp
greg@codemine:~/code/puppet %> cat modules/debsecan/manifests/init.pp
class debsecan {
package { debsecan: ensure => latest }
file {
debsecan:
path => "/etc/default/debsecan",
owner => root,
group => "root",
mode => 644,
source => "puppet:///debsecan/debsecan",
require => Package["debsecan"];
debsecan-cron:
path => "/etc/cron.d/debsecan",
owner => root,
group => "root",
mode => 644,
source => "puppet:///debsecan/debsecan-cron",
require => Package["debsecan"];
}
}
There is really not much to this manifest. It essentially ensures debsecan is installed at the latest available version, it sets up my /etc/default/debsecan config and it ensures there is a cron entry to run it.
Debsecan config
greg@codemine:~/code/puppet %> cat modules/debsecan/files/debsecan # Configuration file for debsecan. Contents of this file should # adhere to the KEY=VALUE shell syntax. This file may be edited by # debsecan's scripts, but your modifications are preserved. # If true, enable daily reports, sent by email. REPORT=true # For better reporting, specify the correct suite here, using the code # name (that is, "sid" instead of "unstable"). SUITE=lenny # Mail address to which reports are sent. MAILTO=root # The URL from which vulnerability data is downloaded. Empty for the # built-in default. SOURCE=
Debsecan cron
greg@codemine:~/code/puppet %> cat modules/debsecan/files/debsecan-cron # cron entry for debsecan MAILTO=root 42 * * * * daemon test -x /usr/bin/debsecan && /usr/bin/debsecan --cron # (Note: debsecan delays actual processing past 2:00 AM, and runs only # once per day.)
You can grab a copy of all the above files (the complete module) here: debsecan-puppet.tar.gz
Using ferm to build firewall rulesets
Posted by Greg in Unix / Linux on December 31, 2010
This post is thanks to a suggestion from JP Viljoen to check out ferm. Well, I did, and it’s fairly neat. You get to express your firewall configuration in structures resembling simple C code along with using things like arrays, functions and if / else constructs which makes building complex rulesets quite a simple task.
I’ve included an example configuration below of one of my machines. The network configuration is not extremely complex, but there is a mix of IPv4, IPv6 and – as this is an IRC server – some DNAT to make the IRC service available on a number of other privileged ports without having the service actually listen on those ports. This particular server is running Debian however ferm is basically just a front to ip(6)tables so it’ll run pretty much anywhere that runs.
First off, here is my network interface configuration to give you an idea of what is where:
kore:~# cat /etc/network/interfaces
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet static
address 173.134.21.19 # Static eth0 IP
netmask 255.255.255.0
gateway 173.134.21.1
iface eth0 inet6 static
address 2001:410:1e9b:ba22::2 # Primary HE.net IPv6 /64 address
netmask 64
auto eth0:0
iface eth0:0 inet static
address 192.168.49.97 # Local networking
netmask 255.255.128.0
auto he-ipv6
iface he-ipv6 inet6 v4tunnel
address 2001:410:1e9a:ba22::2 # Tunnel address
netmask 64
ttl 255
gateway 2001:410:1e9a:ba22::1
endpoint 216.218.224.42
local 173.134.21.19
There is nothing extremely complicated here, just a basic IPv4 static IP assigned by my provider, a local network for traffic between this and other local nodes, a Hurricane Electric IPv6 tunnel and a static IP from my HE.net provided /64.
The ferm configuration in use here looks like this:
kore:~# cat /etc/ferm/ferm.conf
# -*- shell-script -*-
#
# Configuration file for ferm(1).
#
@def $PORTS = (22 25 161 4949 6667 6668 7000 7352 7535); # Services running
@def $IRC_PORTS = (21 23 53 80 110 143 993); # Additional ports
table filter {
chain INPUT {
policy DROP;
# connection tracking
mod state state INVALID DROP;
mod state state (ESTABLISHED RELATED) ACCEPT;
# allow local packages
interface lo ACCEPT;
# respond to ping
proto icmp ACCEPT;
# standard ports we allow from the outside
proto tcp dport $PORTS ACCEPT;
}
chain OUTPUT {
policy ACCEPT;
# connection tracking
#mod state state INVALID DROP;
mod state state (ESTABLISHED RELATED) ACCEPT;
}
chain FORWARD {
policy DROP;
# connection tracking
mod state state INVALID DROP;
mod state state (ESTABLISHED RELATED) ACCEPT;
}
}
table nat {
chain PREROUTING {
# additional ports we listen on and redirect to the IRC server
interface eth0 proto tcp dport $IRC_PORTS DNAT to 173.134.21.19:6667;
}
}
# IPv6:
domain ip6 table filter {
chain INPUT {
policy DROP;
# connection tracking
mod state state INVALID DROP;
mod state state (ESTABLISHED RELATED) ACCEPT;
# allow ICMP (for neighbor solicitation, like ARP for IPv4)
proto ipv6-icmp ACCEPT;
# standard ports we allow from the outside
proto tcp dport $PORTS ACCEPT;
}
chain OUTPUT {
policy ACCEPT;
# connection tracking
#mod state state INVALID DROP;
mod state state (ESTABLISHED RELATED) ACCEPT;
}
chain FORWARD {
policy DROP;
# connection tracking
mod state state INVALID DROP;
mod state state (ESTABLISHED RELATED) ACCEPT;
}
}
So this ruleset is basically broken down into 3 parts:
- IPv4 filter table
- IPv4 nat table
- IPv6 filter table
IPv4 filter table
We control the INPUT, OUTPUT and FORWARD chains here. On the INPUT chain, we default to dropping everything, enable connection state tracking, allow all traffic through our local interface, allow ICMP and specify a list of ports we allow the outside world to use. On the OUTPUT chain we allow everything out and enable connection state tracking. Finally on the FORWARD chain we drop everything as this machine is not a router. Pretty concise right ?
IPv4 nat table
In the nat table config, we basically setup the DNAT of those privileged ports under the PREROUTING chain.
IPv6 filter table
Finally, in the IPv6 filter table, we allow the same set of incoming ports as IPv4, allow ipv6-icmp and setup connection state tracking as before.
Once that’s done, simply run:
kore:~# ferm /etc/ferm/ferm.conf
… and your new ruleset is validated and loaded.
On a side note, if you are interested in playing around with IPv6 I would highly recommend setting up a Hurricane Electric tunnel and then doing the certification. It makes for a great Saturday afternoon time waster and you might learn something along the way:
Getting Git manpages on OS X
Posted by Greg in General, Unix / Linux on April 15, 2010
For some reason the OS X install of Git doesn’t include the manpages. Here is how I installed them.
First off, find the appropriate manpath.
greg@codemine:~ %> cat /etc/manpaths /usr/share/man /usr/local/share/man
/usr/local/share/man looks good…
greg@codemine:~ %> VER=`git --version | awk '{print $3}'`
greg@codemine:~ %> curl -O http://www.kernel.org/pub/software/scm/git/git-manpages-$VER.tar.bz2
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 242k 100 242k 0 0 92051 0 0:00:02 0:00:02 --:--:-- 99k
greg@codemine:~ %> sudo tar xjv -C /usr/local/share/man -f git-manpages-$VER.tar.bz2
Password:
x ./
x ./man1/
x ./man1/git-add.1
[snip]
x ./man7/gitworkflows.7
greg@codemine:~ %> rm git-manpages-$VER.tar.bz2
greg@codemine:~ %>
“man git-add” should now work fine.
Using an alternative mirror for FreeBSD port retrieval
Posted by Greg in Unix / Linux on October 12, 2009
This is something I always search for which doesn’t seem to be very clear from the initial results. The mirrors included below are South Africa specific, so if you are not in South Africa then replace the hostname with something more appropriate for your location.
Add these to /etc/make.conf:
MASTER_SITE_BACKUP?= \
ftp://ftp.za.freebsd.org/pub/FreeBSD/ports/distfiles/${DIST_SUBDIR}/
MASTER_SITE_OVERRIDE?= ${MASTER_SITE_BACKUP}
Mirrors:
| Network | Hostname | Alias | IPv4 | IPv6 |
| Tenet | ftp.za.freebsd.org | freebsd.mirror.ac.za | 155.232.191.209 | - |
| CSIR | ftp2.za.freebsd.org | - | 146.64.8.4 | 2001:4200:7000:1::4 |
| MTN Business | ftp3.za.freebsd.org | - | 196.30.227.198 | - |
| Internet Solutions | ftp4.za.freebsd.org | ftp.is.co.za | 196.4.160.12 | - |
See this post for a far more detailed set of MASTER_SITE make variables. This references a lot more of the locally mirrored content.
Using a ZFS filesystem with Time Machine
Posted by Greg in Unix / Linux on October 4, 2009
This simple how-to explains how to get your Time Machine backups working with a ZFS filesystem. This allows you to use the features of ZFS filesystems for your Time Machine backups.
Please note this is for Mac OS X – Snow Leopard.
1) Enable unsupported network volumes on your Mac by opening a Terminal and pasting this:
greg@macbook:~ %> defaults write com.apple.systempreferences TMShowUnsupportedNetworkVolumes 1
2) Create a new ZFS filesystem and enable CIFS access to it:
greg@opensolaris:~ %> zfs create tank/userbackups greg@opensolaris:~ %> zfs set sharesmb=on tank/userbackups greg@opensolaris:~ %> zfs set sharesmb=name=userbackups tank/userbackups greg@opensolaris:~ %> zfs set aclmode=passthrough tank/userbackups greg@opensolaris:~ %> zfs set aclinherit=passthrough tank/userbackups
You will probably want to setup the correct permissions on your new share, more details in [this post].
3) Make sure you can mount this share and write to it from your Mac.
4) Create the correct disk image:
greg@macbook:~ %> /bin/bash
greg@macbook:~ %> cd /Volumes/userbackups
greg@macbook:~ %> SYSNAME=`scutil --get ComputerName`
greg@macbook:~ %> hdiutil create -size 600G -fs HFS+J \
> -volname 'Time Machine Backups' -type SPARSEBUNDLE "${SYSNAME}.sparsebundle"
greg@macbook:~ %> UUID=`system_profiler | grep 'Hardware UUID' | awk '{print $3}'`
greg@macbook:~ %> cat << EOF > "${SYSNAME}.sparsebundle"/com.apple.TimeMachine.MachineID.plist
> <?xml version="1.0" encoding="UTF-8"?>
> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
> <plist version="1.0">
> <dict>
> <key>com.apple.backupd.HostUUID</key>
> <string>$UUID</string>
> </dict>
> </plist>
> EOF
greg@macbook:~ %>
5) and finally, open up Time Machine. You should now see your network share as an option. Choose it, configure any excludes you want and kick off your first backup!
I’ll post a little later on restoring these backups using one of these methods:
- Restore from Time Machine by using the boot disk
- or by doing a standard install then using the Migration Assistant.
Good luck!
Quick Solaris zone stats
Posted by Greg in Unix / Linux on August 13, 2009
Add this:
alias zonestat="prstat -vZ 1 1 | grep -A50 '^ZONEID'"
to your ~/.profile and you should see something like this when running it:
root@tank:~# zonestat
ZONEID NPROC SWAP RSS MEMORY TIME CPU ZONE
0 58 1198M 1206M 30% 16:15:40 1.7% global
6 25 172M 175M 4.4% 0:03:14 0.0% cl-build
2 27 48M 31M 0.8% 0:00:47 0.0% mirror
Total: 110 processes, 534 lwps, load averages: 0.09, 0.08, 0.07