Posts Tagged ‘CentOS

02
Dec

Server Monitoring.

For these past few weeks I feel like I’ve spent most of my time working on a PCI DSS audit.  It’s the audit which checks whether you’ve made the standards required by the card processing companies - and gives you the ability to store card numbers.  It’s an interesting scheme - with many people having mixed views on it’s suitaility/workability.  I enjoy working to the tough specification, and adding extra bits of security all over.

There’s only so many ways in which you can secure a server.  Most PCI setups involved using multiple boxes (they have to) - and there’s lots of security between the boxes to enhance the security further.  However, the PCI audit doesn’t just check your procedures and systems to avoid the initial penetration - it also looks to migate the affects should some fortunately soul manage to get in.

Due to my background not being in Computing - there are a few areas that I feel I’m weak on.  However, once I can take a look at the problem, it’s normally just a case of logic, that ’supa-doopa’ programming.  In fact, once I’d started programming late in 2008, I realised that the programming was just a tiny part of the process of a programmer - most of the time it was making sure that what was about to be programmed wouldn’t create erroneous results.

One program which I’ve had a love2hate relationship with, that I’ve now really warmed to, is Samhain.  From la-samhna solutions.  It’s a great program, and kudos goes to the developers for releasing it at open source.  I’ve also really enjoyed using SNORT, ntop, wireshark (formerly ethereal) and the Shorewall firewall.  System hardening was an interesting task.. locking the system down to as few users as possible.  There’s so many different things to take into consideration - you end up with a 3D network of traffic streaming from one server to another.

I imagine this as being a gravity-less environment, with streams of data passing like skycars across the 3D network.  I then basically put in roads (the firewall rules) that only allow traffic to be passing

  • on ports I know about
  • transmitting packets I know about (stateful inspection)

Then, when I turn the firewall on - the gravity gets re-enabled - and any datastreams that aren’t supported by the firewall (roads) collapse and are broken.

The most important part of it all though, is the monitoring.  Without effective system monitoring - the whole system is useless.  You need status colours, and a easy 1 screen display.  Events which are expected, such as your developer logging into the server, don’t need to trigger a ‘critical’ error - but can be flagged so the project manager can review them.  Sure, it may be easier for a developer to have plaintext lines, spelling out the status - and then expect the Project Manager to read them - but surely it’s more fun to have a screen of buttons and colours in front of them.  They can immediately see any problems.

But, the main thing I’ve learnt about all this, is that at the end of the day it’s not down to the Sys Admin to secure a system.  If the developers don’t write their code right - if the company policy regarding logins doesn’t prevent changes being made to the live server without PM approval (on pain of death) - then securing it to any level is near pointless.

Needless to say, I’m very happy with what I’ve been doing the past few weeks - and learned many lessons that I think will stand me in good stead for the future.

Bring on the next PCI audit!

30
Sep

“Who do you work for? - My son works for Microsoft; so you’re not a competitor”

In the not too distant past, I read this on Popey’s blog:

Friends wife: “Is that Firefox? Where have I heard of that?”
Me: (not wanting a conversation about geek stuff in the pub) “It’s a product which competes with one of Microsofts products.”
Friends wife: “Nah, we don’t have any competitors.”
Me: “…”
Friends wife: “Except maybe Google.

I thought it was quite amusing, and it’s one of those things that really gets people’s backs up in the Community.  However, less than a week later, I came across the same thing myself.  In my parent’s Kitchen of all places.  One of my Mum’s friends was round, and I was working in Birmingham for the day, so popped home to say hi.  I new her son worked ‘In computers’, and she’s been quite knowledgeable in the past.

Mum’s Friend: “So you work up in Manchester then Andy?”
Me: “Yeah, for a technology company”
Mum’s Friend: “So what do you actually do?”
Me: “I develop Open Source Office Servers & Solutions for SMEs - creating a more affordable system than Microsoft Exchange.”
Mum’s Friend: “Ooh, so is it a big company you work for?”
Me: “No, we have about 20 employees”
Mum’s Friend: “So you’re not really a competitor to Microsoft then… My Son works for Microsoft.”

What defines competitor.. someone who competes in the same market.  I’ve replaced MS Exchange Servers with Open Source Solutions.  If that’s not competing, I don’t know what is!

11
Sep

Value in Diversity

Sometimes we have these moments when something just clicks, and you understand why you believe in something.  One of the biggest criticisms levels at Linux Distros, is that were they to combine, they could share their resources and develop faster.

Whilst I’d hazard a guess at this being true with tribes, and ‘ye olde style’ warfare - in the software world it is exactly this diversity which adds value.  Many groups of people can be coming at the same problem from multiple angles - many of which the other groups have yet to consider as viable/worthwhile.  Therefore you get a variety of methods.

Working on the commercial side of FLOSS, there are a number of Open Source ‘products’ that we re-sell and support.  We add value as a local supplier, and help with end-user support and other IT problems.  The real value we add though, is that we look to support anything (and yes, that includes Vista).  When developing websites, we use a number of FLOSS CMS systems, with each system being a better fit than another for a particular use.  Sure, standardisation may be good on paper… but constantly learning new systems and seeing different approaches adds value to your staff.  Keeping them actively learning stops stagnation, and can only be good for the company in the long run.

07
Aug

Centos5 and ‘yum update’

I came across a ’small’ issue today when writing the policy for our CentOS servers.  It was about how to safely run yum upgrade.. and what happens if something goes wrong.

I have daily backups of the server, so should the worse come to the worst, it’s easy enough to recover… but I was looking for a smarter way.

RPM Rollback seemed to be the answer, however, for the life of my I couldn’t get it to work on my CentOS system.. and as it should have been running since the start, it wasn’t really an option.

What I needed was some kind of system that wrapped up the current rpm binaries, with the configs, and timestamped them so that I’d be ready to revert back to a certain timestamp.  RPM rollback would handle this theoretically.

Luckily, I have multiple levels of redundancy built into the systems, so even if ‘yum update’ does bring down my machine, it’ll have little impact other than irritation, and frustration at a waste of time.

Therefore, for now I’ve simply backed up all the rpms from /var/cache/rpm/*/packages/ into /opt/packages.070808/ ; then ran ‘yum update.’

There’s got to be a better way; please let me know if you know it!

02
Jul

Load Balancing Services on CentOS

This documentation shows how to us ipvsadm to create load-balanced services on CentOS that are capable of handling over 8 million concurrent connections in 1GB of memory. It’s possible to use this to load-balance any service. However, this doesn’t take into account data-syncronisation issues you may incur. For this you’ll need to set up Networked File Systems and Database replication :) (maybe coming soon).

Instructions for CentOS 5 (with ip_vs loaded in kernel as module).

Load ip_vs module

modprove ip_vs

Install ipvsadm

yum install ipvsadm

Run the following commands on the commandline and test them by visiting the $(external_ip) address in your browser.

ipvsadm -A -t $(external_ip):$(port) -s rr -p $(timeout_in_seconds)
ipvsadm -a -t $(external_ip):$(port) -r $(nat_ip_real_server1):$(port)
-m -w $(weight) [ example = 1 ]
ipvsadm -a -t $(external_ip):$(port) -r $(nat_ip_real_server2):$(port)
-m -w $(weight) [ example = 1 ]

Once you’re confident they’re set right.  run:

ipvsadm –save

This will output the config file to the screen.  Copy that into /etc/sysconfig/ipvsadm.

nano /etc/sysconfig/ipvsadm

Restart ipvsadm

/etc/init.d/ipvsadm restart

Test the sites.

Make module & config persistent @ boot time:

echo modprobe ip_vs >> /etc/rc.modules
chmod +x /etc/rc.modules

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!! /etc/rc.modules is used instead !!
!! of rc.local as it loads earlier !!!!!!!!!
!! in the boot process.!!               !!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

chkconfig –add ipvsadm

Done :)

18
Apr

Who reads blogs?

I was having a discussion with a few mates in the pub this evening about my blogging ‘antics.’ They’ve berated me for blogging before, but as it’s becoming more and more widespread I can see them getting more interested in my motivations for ‘blogging.’

One of the friends commented that it was purely the fact that he knew me that made the blog interesting. For someone that didn’t know me, the blog would be pretty dull and of no consequence. At this point, another chipped in saying - “Only bloggers read blogs.” Is this true? I don’t know, but I don’t think so.

My motivation for starting the blog was that it was a place where I could share my technical insights. Not profound insights such as the advent of structural-object-abstract programming methods that I’ve just decided are going to be the Web 3.0 - but short howto’s and the like, and to publish a few ‘Gotchas’ - problems that have few symptoms and a nice easy solution - but take hours of work to solve. I blog advice - There’s nothing quite comparable experience.

(un)Fortunately, which ever way you personally look at it - my blogging has branched out to cover all sorts of things. From the time when some guy smashed the window of my car, to re-living and walking through my car accident in 1999, to a short article on why to avoid Red Hat’s bundled openLDAP implementation because it’s crap.

I think it’s an interesting concept, for non-bloggers and bloggers alike. Who reads blogs? If you have a regular commentator on your blog, do you add him to your blogroll as a thanks for lifting your self-esteem by having him visit your blog? Do you think you have a regular readership, or just random visitors popping in and out after being directed from Google?

I don’t think it’s a negative thing that bloggers read blogs. It’s great. From the attendance at the spontaneous meet-up last Friday, it’s clear to me that there’s a nice little community of bloggers in Birmingham. However, this is a meeting of a cross-section of the readership who it’s worth meeting face-to-face in order to better your own blog.

Is blogging journalism? Is it art? Is it a cry for help from some pathetic moron wanting to share his story with the world? Is it ‘new media’? Does it matter?

I blog tech because I think some people read it and it helps them - and also as an easy reference for me. I blog ‘about me’ as a way to vent some thoughts and get some feedback from an audience who I think would be interested. This audience is dynamic, therefore I categorise my posts different to respect that.

When doing my Psychology degree, one of the things we covered was ‘online personalities and freedom of information.’ This blog is in the public domain. If it were a diary, having it leaked would immediately bring headlines of ’scandal and gossip.’ My blog is sort of a base for my online identity. ‘andylockran’ lives here. I happen to pop up on a mailing list or a forum or IRC and you want to know more about me. Much of it is here. It’s a bit like ‘CV 2.0.’ The web is my field - if I don’t market myself well on the web, how the hell can I expect anyone to be able to trust me to market their products on the web?

The best thing about it for me is the feedback, both positive and negative. Setting up a blog exposes you to both - and it gives you the opportunity to have a voice.

07
Feb

Home Desk

I’ve moved back home and have just set up my desk. I quite like what’s going on at the moment therefore I thought I’d share a couple of pictures.

Other Desktop View
I have my fishtank to the left, then a mac mini running OSX Tiger - useful for checking cross compatibility. Then I have my sound-dock which is connected to my PC via my “unique connector.” I then have my Webcam, followed by my 19″ Dell Monitor running Gentoo & my virtual machines (Windows and Centos). Centos I use as it’s my company’s choice of server distribution, and Windows for cross-platform testing and to connect to my o2 XDA Stellar (shown under the screen). I also have my work VoIP phone, which is very useful, and my HP Compaq nc4000 lightweight notebook running ubuntu.

imgp0863.JPG

All in all a quality set-up.

21
Dec

Documentation

In a recent blog I wrote about how Meccano deliberately sabotaged the instructions to make kids use their intuition to solve problems.  Well I think it’s only fair that if we hit problems we share the solutions.

Therefore, my solution today is the following:

Make sure the feature you are trying to implement is present in the release version of the software you are actually using.

It may not be the first time I have made this mistake - but I hope it’s the last.