Hip Hip Array

Setting up the wonderful HAProxy

Fri, 02 Mar 2012

As mentioned in a previous post we love HAproxy at Pixl8 its the handy penknife we use for high availability solutions.

1. TCP/IP balancing for SMTP relay,

2. High availability for MySQL

3. High availability for membership websites

4. Balancing load across our many varnishcache caching servers.

Below is a simple step by step how to assuming you have a Centos or Redhat Enterprise linux server, I haven't covered firewalling that should be handled seperatelty and I assume the readers comfort with Linux in this regard.

Once you're at the linux shell as root perform the following

type make

If no make is installed yum install make

cd /tmp
wget http://haproxy.1wt.eu/download/1.4/src/haproxy-1.4.19.tar.gz
tar -zxf haproxy-1.4.19.tar.gz
cd haproxy-1.4.19
make "TARGET=linux26 USE_PCRE=1"
mv haproxy /usr/sbin/
wget http://www.pixl8.co.uk/downloads/haproxy.init.txt
mv haproxy.init.txt /etc/init.d/haproxy
chmod +x /etc/init.d/haproxy
chkconfig --add haproxy
chkconfig haproxy on
wget http://www.pixl8.co.uk/downloads/haproxy.cfg.txt
mv haproxy.cfg.txt /etc/haproxy.cfg
service start haproxy

You will have a standard config that is proxying www.cnn.com, you can test this by adding www.cnn.com to your hosts file and pointing it at your HAProxy server IP of course need to edit the haproxy config file to reflect your server environment but the config piece is straightforward, make sure you restart haproxy on config changes using

service haproxy restart

There is a console set to listen on port 8080 the password details are in the sample config if you use it you should change them to your own values.

Open Bluedragon 2.0 is released!

Sat, 19 Nov 2011

Open Bluedragon ?

Pixl8 has been an avid user of ColdFusion since the late 90's when I used an early version for my University Dissertation. Since then we've gone through many versions and I used to train the formal Macromedia now Adobe certified curriculum. Back in 2005 we started using alternatives Bluedragon.net and now we're a massive fan of Open Bluedragon.

Whats special about version 2.0 ?

Open Bluedragon 2.0 has finally been released and we've been using the latest bleeding edge nightly builds in some specific cool projects for some 6 months now. The Open BD team has really pulled out all the stops and delivered some very elegant features.

Some of my particular favourites are

1. CFSMTP (Write Mail based applications that process Mail and respond accordingly, we'll be sharing details of how we're leveraging this killer feature soon!)

2. CFMAIL call backs for better mail handling

3. Awesome deep json support

4. Server-side JQuery style selectors using the JSOUP libraries and exposed as a simple HTML() function

5. Server side inline Javascript <cfscript language="javascript">

6. Server side inline Java <cfscript language="java">

These last two features means we will now be able to offer developers to write their CMS templates in CFML, Serverside Javascript or Java. I'm currently experimenting with Quercus which would allow us to also provide PHP support.

For more information check out the blog post from Alan Williamson or head over to the project site

HAProxy is awesome!

Sat, 10 Sep 2011

I thought I'd share some information on one of the technologies we use for our larger sites HAProxy.

I know many agencies naturally try and avoid getting involved in Infrastructure but I'm a believer in understanding all elements of the stack that you build your applications on, the i'll just shove it in the cloud works in theory only.

So when we needed a solution to do load balancing recently rather than go and buy the latest piece of Cisco kit or use our standard KEEPALIVED setup I hunted around.

The interesting challenge was that we wanted to do more than simple IP based load balancing, we needed something which was essentially application aware. Having looked at Zeus which we used back in 2006 for a large University client and discounted it I looked at HAPROXY.

HaProxy is essentially a reverse proxy than can do either TCP or HTTP based load balancing, its super fast and very elegant. We run it up on Centos on some moderate VMs and it hums away using next to no resources.

You can learn more about it here http://haproxy.1wt.eu/

There is a useful monitoring tool here http://feurix.org/projects/hatop/

Needless to say we have it in production for a few clients including a new Social networking startup.

My particular favourite feature is the ability to do rule based sticky sessions which for example only take affect after a user has done a post or based on the existance of certain cookies.

HAProxy plays very nicely with VarnishCache and Ngnix which I'll blog about another time.

.NET request size gotchas

Mon, 18 Jul 2011

Thought this was worth blogging as it's a real gotcha in .NET. Like PHP there is a default file size specified which you need to tweak to allow big file uploads. Unlike PHP, .NET fails silently and also doesn't point out the other settings which need tweaking like request size.

Below is a sample web.config which gives a much bigger upload size. Useful for allowing those big uploads to the server even if these days we seem to pump it out to a CDN like Akamai as fast as we receive it.

web.config

Also a useful tip. Any changes or saves to the web.config will automatically result in the application being restarted which is nice, though may cause you to be logged out if you're not using out-of-process sessions.

SQL Server analysis

Sun, 10 May 2009

I've been looking at analysing sql server query usage and have used SQL profiler to ascertain long running queries and generally log interaction with the DB.

Here is an interesting URL on the subject

http://www.developer.com/db/article.php/10920_3482216_1

in addition I was looking for a way to do this using a query this only works on SQL Server 2005, i found this query (i forget where) but i modded it so that the actual debug query is excluded from the results.

/*Debug query */
SELECT TOP 100
qt.text as QUERY,
qs.execution_count,
qs.total_logical_reads, qs.last_logical_reads,
qs.min_logical_reads, qs.max_logical_reads,
qs.total_elapsed_time, qs.last_elapsed_time,
qs.min_elapsed_time, qs.max_elapsed_time,
qs.last_execution_time,
qp.query_plan
FROM sys.dm_exec_query_stats qs
CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) qt
CROSS APPLY sys.dm_exec_query_plan(qs.plan_handle) qp
WHERE qt.encrypted=0
/* Filter this query out */
and qt.text not like '/*Debug query */%'
/* Filter out any system level calls made by the GUI */
and qt.text not like '%FROM sys.%'
/* Reduce analysis window to today */
and year(last_execution_time) = year(getdate()) and month(last_execution_time) = month(getdate()) and day(last_execution_time)=day(getdate())
ORDER BY qs.total_logical_reads DESC

Getting table information on MySQL

Tue, 12 Feb 2008

Dissecting a database again the joys of trying to work out what talks to what.

Some time ago I blogged a post about finding table information from sql server well the query below has proved useful today in working out what tables are used as part of the application i'm analysing.

SELECT TABLE_NAME,TABLE_ROWS,UPDATE_TIME AS LASTUPDATED FROM INFORMATION_SCHEMA.TABLES
WHERE table_schema = 'MYDATABASENAME'
ORDER BY UPDATE_TIME desc

Checking tables on SQL Server

Thu, 15 Feb 2007

I was reviewing a SQL Server database application the other day which had 100s of tables many of which I was pretty sure were not being used with the absence of any documentation i needed a way of listing all the tables and then finding out whether or not they contained data.

The following query did the trick and works on SQL Server 2000 and 2005

select so.name as tablename,
Max(si.rows) as Rows
FROM sysobjects so,
sysindexes si
WHERE so.type='U'
AND
si.id=OBJECT_ID(so.name)
GROUP BY so.name
ORDER BY Rows DESC

Update: Very useful, I've already lost track of how many times i've used this.

The difference between variables.instance and instance

Mon, 29 Jan 2007

Trouble shooting some bung code I came across a weird one today. This highlights the need to scope variables.

Quite often for cfc's when storing private instance data developers use a structure called instance under the variables scope or private. You can of course just use variables, but anyway it seems a trend in some code i've been reviewing.

Anyway I suppose it's better than using the this scope as it means the data is private to the cfc instance.

There is however a difference between variables.instance and instance, the test case I created below shows how lazyness can byte you in the arse.

this.cfc --------

test.cfm --------

<cfset testCFC=createobject("component","test").init(datasource="foo")>
<h1>Having a form field named instance breaks my cfc</h1>
<form name="thoughshallscope" action="" method="post">
<input type="hidden" name="instance" value="oh dear">
<input type="submit" name="gostuff" value="break my cfc">
</form>
<br>
<a href="?instance=KillthatCFCYeah!!">Having a link with the word instance in isn't good either</a>

This can be fixed by modifying the init method firstly to use variables.instance but secondly to make sure you use structNew() before working on a structure.

Slow MySQL Server connections

Fri, 05 Jan 2007

We had a problem recently with slow connections to a MySQL 5 database on linux. The symptoms were that for the first connection from the client (cfmx /bluedragon/query tool) the response from the MySQL server would be very slow up to 15secs. Then after that the reponse was quick (This was actually a bit random under load with connection pooling).

It turns out that if you experience slow connections its likely that your MySQL server is trying to do a reverse DNS lookup against the client and does not have access to a DNS server.

Now there is an option that can be set for MySQL --skip-name-resolve which will solve this if you cannot give it a DNS Server but for me adding a local DNS fixed the issue (Make sure you restart MySQL for it to pick it up though)

The MySQL command

show processlist

Is invaluable in diagnosing these issues as it'll show current connections, it's similar to the SQL server equivalent

sp_who

Connecting local drives on terminal services

Wed, 03 Jan 2007

When using remote desktop to access Windows servers one of the common complaints is the lack of built in file transfer client like SFTP with SSH. I discovered recently that you can connect local drives and they appear on the remote server (Windows 2003) (Thanks to Paul Overton for this tip). To do this on the connection screen click

Options --> Local Resources --> And ensure disk drives is checked. You can also uncheck the printers unless you want that which prevents the annoying event log messages on the server.

Coldfusion and Java Tuning link

Fri, 29 Dec 2006

I moved this from the resources section of the Pixl8 Interactive web site as now we have an externally facing blog it makes more sense for these links to be here.

JRun 4.0: Configuring individual JVM settings for each JRun Server

The Moment of Truth: Tuning Performance in JRun Apps and Environment

JRun 4: General troubleshooting techniques for an unresponsive JRun server

ColdFusion MX: Explanation of Request Timeout and threadWaitTimeout settings

Server settings: the jrun.xml file

ColdFusion MX: Tips for performance and scalability

Frequently Asked Questions about Garbage Collection in the HotspotTM JavaTM Virtual Machine

Tuning Garbage Collection with the 1.4.2 Java[tm] Virtual Machine
(see also Peter Freitag's summary)

Java Tuning White Paper

jvmstat 3.0

ColdFusion Peformance Tuning Presentation

Obtaining a Thread Dump with ColdFusion or JRun running as a Windows Service

JRun Connection Closed Errors

Performance Tuning and Monitoring Coldfusion or JRun

CF_Hypercache

Java Thread and How does it work?

Advanced Debugging In JRun 4

Steven Erat's blog: activeHandlerThreads or Simultaneous Requests: Less is More

Simultaneous Requests in CFMX 7

Keep Library Loaded - Check this box to retain the library in RAM

SeeStack - Parse ColdFusion Thread Dumps Like a Pro!

Request timed out waiting for an available thread to run

Troubleshooting Tips For ColdFusion MX

Help! My ColdFusion MX Server is Slow!!!!

Sarge's blog: CFMX Performance Settings (JRun 50% CPU thread)

Robi Sen's blog: Hung Servers, Java.lang.OutOFMemory errors and Tuning CF JVM

Peter Freitag's blog: Tuning Garbage Collection Outline

ColdFusion Garbage

ColdFusion Memory Usage Stats

Getting ColdFusion Server Metrics

Have you tuned your JVM on ColdFusion MX yet?

CFMX JVM Tuning - The difference between MaxPermSize and Heap Size

Daemonite blog: Maximum JVM heap size for CFMX

Enhancing CFMX logging

Misc: Java Garbage Collection Tuning

Java Performance Tuning

SeeFusion :: ColdFusion Server Monitoring