I'm speaking from XF 4.1 carb ULP knowledge.
If you want quick acceleration, put a 3.23 or higher diff in, assuming you've got 2.77 diff. You'll need to pay someone to do it (about $400 - $450 is best I could find in Melbourne) unless you've got special tools and skills and experience, or pick up an axle disc to disc (or drum to drum if that's what you've got) and swap it over. Check the ratio by removing the diff cover and checking the ratio printed on the crown wheel. The tag on the diff cover means nothing - some prick in the Trading Post tried to sell me a 2.92 as 3.23 for my XF a while back. I think XE's all had coil rear axles. If so, check that the axle matches yours for coil mounting points, Watts link etc - I don't think XD, ute, and panel van axles for leaf springs can fit.
Intake on 4.1 alloy head cross flow head is good, but intake manifold is too narrow and can't flow to head's potential. Still some Cain manifolds new at John Cain in Sydney for $140 a couple of months back, but set up for 350 Holley, as I think are all other performance manifolds. If you've got two barrel Weber (I'm not sure what was on XE) it will flow about same as Holley with no problems with emission / roadworthy regulations. Adapter plate for Weber can be made from aluminium plate with drill and hacksaw and file.
Exhaust on 4.1 not as good as intake, but no point doing head work as you're on a budget and no point unless you want serious performance improvements higher in the rev range. Extractors definitely and bigger exhaust and better mufflers if you can afford them.
Disable the EGR system, but leave it in place for roadworthy etc. Buy two EGR to manifold gaskets for a few bucks. Assuming EGR mounting is same as my XF, remove EGR from intake manifold and make plate out of thin aluminium, using a gasket as template. Sandwich plate between two gaskets and reinstall EGR. No more exhaust gases to dilute the fuel/air charge. And it can be removed in a few minutes if you ever get pinged.
Keep vacuum advance dizzy as allows recurving to get maximum out of other changes, unlike XF EST box.
Decent cam (look at Crow cams website for choices) and heavier springs will improve performance and increase redline 500 rpm +.
If you're XE you'll be leaded fuel, but all alloy heads have to have hardened valve seats or they'd be stuffed in no time, so you should be ok with unleaded. If you shave head or alter pistons to increase compression, you'll have to be careful to avoid pinging, but if you can afford to run Optimax it'll easily go to 9.5:1 (according to Shell anyway).
The fan soaks up anything between 5 and 10 BHP and is noisy to boot. You can get a couple of thermo fans off any front wheel drive (about $30 each at Pick a Part) and fit them, but you'll need to fit a thermo switch as well.
5 sp box is good if T5, but expensive. Reco ones are getting up around a grand, and that might only be if you've got one to change over. BW 0507 won't last in 4.1, and won't last even quicker in modified 4.1.
4sp single rails a lot more plentiful and cheaper in Trading Post, but I'd leave gearbox till last of all the suggestions in this topic. You might need to change tailshaft and/or front yoke to fit different gearbox. It's the most expensive item for the least result under the foot.
