Magento dataflow batch exports and parent SKUs of configurable products

At work, I've been scratching my head how to easily batch export and import Magento products. Importing is covered pretty well by <a href="http://sourceforge.net/apps/mediawiki/magmi/index.php?title=Magmi_Wiki">Magmi</a>, but the exports are a bit lacking. Found an extension to add extra abilities to the Dataflow export, including category exports, in here - http://www.magentocommerce.com/boards/viewthread/60113/ - but the one thing I wanted was to be able to export the parent SKUs of complex products, as Magmi can handle them in their own field (simple_skus). However, I then realised I'd done it completely the wrong way around, and I actually needed child products. So I've extended the extension, and thrown it up on GitHub, hopefully it'll save someone else the hassle for either purpose. Seems to work well, but let me know if there's any issues, I'll see what I can do to help.

Usage is the same as the original extension, with the addition of a few options at the bottom:

<action method="unparse" type="exportextension/modifier">

    <var name="remove_line_breaks">true</var>

    <var name="remove_html_tags">true</var>

    <var name="add_categories">true</var>

    <var name="category_field_name">category</var>

    <var name="category_delimiter">#</var>

    <var name="category_first_level">-1</var>

    <var name="category_path_delimiter"><![CDATA[>]]></var>

    <var name="add_absolute_url_to_field">product_url</var>

    <var name="add_image_url_to_field">image_url</var>

    <var name="add_parent_sku">parent_sku</var>
    <var name="add_child_sku">child_skus</var>
    <var name="child_sku_delimiter">,</var>

</action>

- remove_line_breaks: if the value of this tag is “true”, linebreaks will be removed from the content 

- remove_html_tags: if the value of this tag is “true”, html tags will be removed from the content 
- add_categories: if the value of this tag is “true”, each product will be extended with category names. 
The following tags, are only used, if “add_categories” is “true”. They configure the additional category field: 
- category_field_name: this value defines the used name of the category field. The default value of this tag is “category”. This tag is only expedient, if you export the fieldnames aor if you allready have another field with the defaultname “category”. 
- category_delimiter: if a product is at more than one categories, each category will added and they will be delimited by this string. The default value is “#”. 
- category_first_level: this value defines the level of the first exportable category. If it is 2 or greater, the first category names of the category tree won’t be added to the data. The values 1 and 0 are the same, because there is no level 0 - the root category has level 1. If the value is set to “-1”, only the proper category name will be added to the product. The default value of this tag is “1”. 
- category_path_delimiter: if the category paths are exported (category_first_level is not -1), each category at the path will be delimited by this string. The default value is “>”. Example: catA1>catA2#catB1>catB2>catB3 

- add_absolute_url_to_field; wit this tag you can set a field name, which will be used to add the absolute product link. if this field does not exists, it will be added. if it exists, it will be overwritten. 
- add_image_url_to_field; this tag works at the same way, but exports the absolute image link.

- add_parent_sku: If this field is added, the parent SKU of any simple products associated with a configurable product will be added as a field.
- add_child_sku: If this field is added, the child SKUs of any configurable products will be added as a field. If intended for use with Magmi, this should be named as simple_skus.
- child_sku_delimiter: Delimiter for the add_child_sku field, default if unspecified is a comma.


Limitations: This will only currently pull up the first parent SKU of a simple product, if it is assigned to multiple configurables. I didn't look into that any further at the time. This does not affect child products of configurables, which will be properly seperated by the delimiter specified.

I've uploaded the source to GitHub,at https://github.com/microchip/Magento-ExportExtension - both the main extension code, and a prepackaged version in the package folder. Hope it helps save someone the pain I went through trying to figure this damn thing out. The only answer you seem to find online is "Well, if you're struggling with this, here's an extension we'll sell you to fix the issue!" Open source at its best, huh, guys? Don't get me wrong, I understand the desire to write extensions and make money, but the amount people seem to charge is a bit ridiculous, especially if you only want one feature out of the 10 the extensions that are being peddled do.

Anyway. Rant over...

Solution to audio on the Toshiba Satellite Pro P100 and Windows 7 64-bit muted on install

I've just had a seriously non-fun few hours arguing with drivers on the Toshiba Satellite Pro P100 (specifically the PSPAEE-00E006EN variant). Had no real issues installing Windows 7 64 bit, didn't really expect to as Vista 32-bit worked perfectly well. 135 updates installed later, including the graphics driver, everything seemed to be working swimmingly.

Except the sound, which while it pretended to play, it clearly was not.

Apparently I was far from alone in this issue. Many forums complained about lack of Vista x64 / Windows 7 x64 failing to work properly with the sound. I tried installing a variety of sound drivers, none of which seemed to actually work, although strangely occasionally I'd try to install one, it'd fail, but the sound would work until I rebooted. Extremely frustrating.

Eventually, after much hunting down and forum crawling, I found this driver from HP - SP40170 - which did actually work. The automatic install program didn't, but installing the driver manually via device manager - update driver - select from a location on disk, and choosing c:\SWSETUP\SP40170\Venice\V64 allowed me to install a working driver, and get sound.

It's not perfect - the sound stops working on suspending the machine, which I haven't figured a solution for yet aside from reboot - but other than that it seems to be working well so far. I'm a bit surprised (and disappointed) that a proper 64-bit driver wasn't released for Vista/7 by Toshiba, but hey, just the way the cookie crumbles with older hardware I guess.

Hope this saves someone else the hours I've had to spend hunting this down.

FreeNAS, Linux, ZFS, and a lesson in debugging slow file transfers...

A while ago, I had my Proliant N40L server running as my home NAS, on FreeNAS 0.7, which was working rather well as a file server. Didn't have much fun getting the DLNA streaming working properly, but the ZFS implementation was great, the speeds were rapid, and it was quite brilliant. Unfortunately I also used 4x 1TB Seagate Barracuda desktop drives in it, which resulted in one dying after a while with the vibration in the cage. Oops.

So, one rebuilt server later with 4x 3TB WD Red NAS drives, I was ready to build a new tank. Decided to have a go with FreeNAS 8.2, which worked quite well. However, don't be fooled by the out-of-box configuration and built-in tweaker - I had to drop in quite a few optimisations from my old 0.7 setup to get it running quickly. But it was running quite well, it has to be said. There's 8GB of RAM in this box, for context later.

However, after some thought, I decided I wanted to get the Plex Media Server running. Sadly, at the time of writing this, it doesn't support FreeBSD, nor can FreeBSD run it under the Linux emulation as it uses epoll, which isn't supported at the moment on the BSD side. So... I decided to have a go with Linux.

One thing I didn't want to lose from the FreeNAS implementation, which was one of the primary reasons for using it, was ZFS. A while ago I'd discovered ZFS on Linux, a full kernel module for Linux, which neatly sidesteps the licensing issue (ZFS is licenced under the old Sun CDDL, which is incompatible with GPL, which is why ZFS isn't happily in the Linux kernel and doing amazing things). So, installed a copy of Ubuntu 12.04 server onto a USB flash drive via a VM on my PC, booted the microserver up off the USB key, and added the repositories for ZFS on Linux. A couple of quick installs later, and lo and behold, ZFS on Linux. Surprisingly, the performance outdid both FreeNAS implementations, although that could be other factors at work at this point:

FreeNAS 0.7

freenas:/mnt/tank# dd if=/dev/zero of=./testfile bs=1M count=10K
10240+0 records in
10240+0 records out
10737418240 bytes transferred in 87.457035 secs (122773637 bytes/sec) - 117MB/sec write
freenas:/mnt/tank# dd if=./testfile of=/dev/null bs=1M
10240+0 records in
10240+0 records out
10737418240 bytes transferred in 60.337002 secs (177957437 bytes/sec) - 169MB/sec read

FreeNAS 0.8 (before tweaks)
[root@freenas] /mnt/storage# dd if=/dev/zero of=./testfile bs=1M count=10K && dd if=./testfile of=/dev/null bs=1M
10240+0 records in
10240+0 records out
10737418240 bytes transferred in 225.523565 secs (47611070 bytes/sec) (45.4MB/sec)
10240+0 records in
10240+0 records out
10737418240 bytes transferred in 78.246473 secs (137225588 bytes/sec) (131MB/sec)

FreeNAS 0.8 (after tweaks)
[root@freenas] /mnt/storage# dd if=/dev/zero of=./testfile bs=1M count=10K && dd if=./testfile of=/dev/null bs=1M && rm ./testfile
10240+0 records in
10240+0 records out
10737418240 bytes transferred in 62.405507 secs (172058825 bytes/sec) - 164MB/sec write
10240+0 records in
10240+0 records out
10737418240 bytes transferred in 53.998347 secs (198847165 bytes/sec) - 189MB/sec read

And finally, Ubuntu 12.04 with ZFS on Linux:

root@RickNAS:~# dd if=/dev/zero of=/storage/testfile bs=1M count=10K && dd if=/storage/testfile of=/dev/null bs=1M
10240+0 records in
10240+0 records out
10737418240 bytes (11 GB) copied, 67.544 s, 159 MB/s
10240+0 records in
10240+0 records out
10737418240 bytes (11 GB) copied, 31.78 s, 338 MB/s

rick@RickNAS:~$ dd if=/dev/zero of=/storage/testfile bs=1M count=10K; dd if=/dev/zero of=/storage/testfile2 bs=1M count=10K; dd if=/storage/testfile of=/dev/null bs=1M; dd if=/storage/testfile2 of=/dev/null bs=1M

10240+0 records in

10240+0 records out

10737418240 bytes (11 GB) copied, 61.1301 s, 176 MB/s

10240+0 records in

10240+0 records out

10737418240 bytes (11 GB) copied, 59.9313 s, 179 MB/s

10240+0 records in

10240+0 records out

10737418240 bytes (11 GB) copied, 47.6692 s, 225 MB/s

10240+0 records in

10240+0 records out

10737418240 bytes (11 GB) copied, 32.2975 s, 332 MB/s

Quite strange, the improvement, but I wasn't going to knock it. So, all looked good, until I tried doing file transfers over CIFS (Windows networking), FTP and SFTP. The results were... abysmal. CIFS and FTP started fast, and rapidly dropped to about 30MB/sec. SFTP was... well, 3MB/sec didn't impress at all. To ensure that ZFS wasn't the issue, I reformatted one of the partitions as ext4, and did another test:

root@RickNAS:/storagetest# dd if=/dev/zero of=./testfile bs=1M count=10K

10240+0 records in

10240+0 records out

10737418240 bytes (11 GB) copied, 66.944 s, 160 MB/s

root@RickNAS:/storagetest# dd if=./testfile of=/dev/zero bs=1M

10240+0 records in

10240+0 records out

10737418240 bytes (11 GB) copied, 68.2629 s, 157 MB/s

root@RickNAS:/storagetest#

Not as fast as the raidz was, but still reasonably respectable. Sadly, I got similar results from CIFS etc.

Deciding to stick with ext4 for the moment just to eliminate any extra possible issues, I started to wonder why the performance was so terrible. I ran iperf to my PC, and got pretty respectable speeds out of it, far exceeding the data speeds I was getting in transfer. I went to work trying to improve the Samba speed, to no avail. No matter what I tweaked, nothing seemed to have any effect.

Next step: attempt reinstall natively, after having written out the ISO to USB.

So, reinstalled, after using the PenDrive Linux installer. Got it installed natively this time. Made zero difference. SFTP speeds are pathetic, around the 10MB/sec mark. Across gigabit, that's a bit of a joke, and it's nowhere near maxing CPU, so it's not that as far as I can tell. Did read up online that the driver for the onboard NIC can sometimes be less than perfect and to upgrade to a newer kernel. Did that. Nothing happened. Well, time passed, that's about it. iperf figured look okay though, averaging about 850mbit mark, which isn't too shabby. So there's some sort of bottleneck going on, and it's getting quite frustrating.

Next, I've tried to download /dev/zero via SFTP (in FileZilla on the Windows side) and it's settled for now at about 7.2MB/sec download speed. There's something quite clearly wrong. It's using 36% of one CPU for SSHD and 4-5% for sftp-server. It's clearly quite broken. I created a 4GB file in tmpfs in RAM for transfer tests, and vsftpd has slowed to 6.6MB/sec, which is very broken indeed. Make that 6.1MB/sec. Ouch. Rerunning the SFTP transfer resulted in SSHD taking up 96% of CPU at first (fair enough, maxed out) and doing about 40MB/sec, but it rapidly dwindled back to the 8MB/sec mark, using 26% CPU.

Next step: trying local FTP. Hooked in via localhost, retrieved said ram-based file and wrote to disk. 4294967296 bytes received in 26.49 secs (158306.3 kB/s). So, 26s to download the 4GB file, 157.5MB/sec. Seems reasonable. The put was even faster, 13.58 secs for the 4GB. So clearly it's not the software, or at least, not when running locally.

Okay. Back to the basics. Ran a few more longer iperf tests, for 6 minutes at a time each way, speeds were around 820mbit/sec on average. Nothing too spectacular, but nothing too unreasonable, and I was using the PC that they were connecting to at the time as well. Out of curiousity, I installed the CIFS utils and mounted up my Windows network share onto Linux, to copy some files across via the protocol to see what happened. This is where it got a little more confusing.

First, I ran DD to test it, using an mkv video file, 1.6GB. The second time I ran it after dumping the cache, just to make sure, was a little slower than the first time but still fairly respectable.

root@RickNAS:/mnt/win/Rick/Downloads# dd if=./testfile.mkv of=/tmpfs/testfile.mkv bs=1M

1548+1 records in
1548+1 records out
1623222074 bytes (1.6 GB) copied, 16.5486 s, 98.1 MB/s

98MB/sec. Reasonable enough, I think it came out at about 105-110 last time. The same timed operation with cp, after a reboot to flush caches, and a different file of similar size at 1.53GB (as I was struggling to kill caches):

root@RickNAS:/mnt/win/Rick/Downloads# time cp ./testfile.mkv /tmpfs/testfile.mkv
real    1m25.174s
user    0m0.096s
sys     0m6.992s

So, 85 seconds to copy 1.53GB, which equates to just under 18.5MB/sec. Seems a bit harsh. Dropped the cache, and ran it again to see what happened, after the Windows box had cached it:

real    0m17.701s

user    0m0.096s

sys     0m7.248s

So. No real problem copying there, at 88.5MB/sec. Similar to the above then, really, but a little slower. Still, acceptable enough. At this point, I have to eliminate the Windows box as a potential source of slowdown. Theoretically it shouldn't be, as it's on a RAID 0 with a pair of 500GB drives, but... one way to find out. Cue DD for Windows, and a /dev/zero device driver (translated: \\.\zero):


C:\temp>dd if=\\.\zero of=.\testfile bs=1M count=1K
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 11.5837 seconds, -0.0 MB/s

Well, that -0.0 MB/s is wrong, but the theory is reasonable. Trying a 10GB file for good measure:

C:\temp>dd if=\\.\zero of=.\testfile bs=1M count=10K
10240+0 records in
10240+0 records out
10737418240 bytes (11 GB) copied, 812.172 seconds, -246489356887425550000[snip] MB/s

Erm... 0.21MB/sec?! I know I did a bit of web browsing in the meantime... but there's something wrong there. I'm somewhat hoping it's the /dev/zero driver...

Okay. So plan B. Let's try a dd from the Linux box to the Windows box, the other way around.

root@RickNAS:/mnt/win/Rick/Downloads# dd if=/tmpfs/testfile.mkv of=./testfile.mkv bs=1M
1576+1 records in
1576+1 records out
1653435119 bytes (1.7 GB) copied, 123.167 s, 13.4 MB/s

... This would be a facepalm sort of moment then.

So what the hell is going on with the Windows box?!

CrystalDiskMark didn't report anything too strange on sequential read/write. 178.8MB/s read, 102.8MB/s write.

After this, I was entirely unable to access any of the mounted files on the Windows side, which left me rather confused. Had to drop this fix into my Windows box, and then all was shiny again. Very odd. So, another try of the DD then...


dd if=/tmpfs/testfile.mkv of=./testfile.mkv bs=1M
1576+1 records in
1576+1 records out
1653435119 bytes (1.7 GB) copied, 60.3011 s, 27.4 MB/s

Slightly better, but still a bit rubbish, in all honesty. Again, for good measure:

Resource monitor did show the write speed at 75MB/sec... briefly. Oh.

1576+1 records in

1576+1 records out

1653435119 bytes (1.7 GB) copied, 18.7947 s, 88.0 MB/s

... I don't get it. Same again got 87.3MB/s. It shows it as writing. Okay. Let's try another FTP download shall we? Nope, still rubbish.

Okay, time to eliminate the hard disk as an issue. Time to download to a USB HDD and see how that goes.

... Not so well. It chugged along at about 11MB/sec. That's quite poor. But it was fairly consistent, at least. CrystalDiskMark gave it a whole 10MB/sec write too, so no shock. Okay. Throwing a pure RAM disk into Windows to take out the drives as culprits.

dd:

root@RickNAS:/mnt/win# dd if=/tmpfs/testfile.mkv of=./testfile.mkv bs=1M
1576+1 records in
1576+1 records out
1653435119 bytes (1.7 GB) copied, 16.2628 s, 102 MB/s

Looks far more like normality.

ftp:

File transfer successful, transferred 1,653,435,119 bytes in 16 seconds.

So, that'll be... 98.55MB/sec then. Oh good.

Out of curiosity, I ran sftp, just to see what the performance was like, although I was expecting "dire". I got what I expected, about 20MB/sec. Fair enough, that's CPU-bound. Okay, other way round, just to make sure:

root@RickNAS:/mnt/win# dd if=./testfile.mkv of=/tmpfs/testfile.mkv bs=1M
1576+1 records in
1576+1 records out
1653435119 bytes (1.7 GB) copied, 16.1864 s, 102 MB/s

Looks good. Trying cp commands both ways:

root@RickNAS:/mnt/win# time cp /tmpfs/testfile.mkv ./testfile.mkv

real    0m16.803s
user    0m0.064s
sys     0m7.208s
root@RickNAS:/mnt/win# time cp ./testfile.mkv /tmpfs/testfile.mkv

real    0m15.218s
user    0m0.060s
sys     0m7.432s

So. RAM to RAM is actually fine. I'm going to run Samba to test it out too, to make sure all is as it should be...

... From: 100MB/sec. Copying back 110MB/sec.

Oh.

Okay, so let's try Linux HDD to ramdisk, just to make sure all is shiny as it should be.

... About 110 each way.

So. After all that hunting... all that debugging... all that chasing ghosts, upgrading network drivers and kernels, tearing my hair out... there was nothing wrong with the Linux box at all... it turned out it was my local hard disk. I just hope that this post saves someone else the hours of attempted fixes I tried. It's been a painful journey, but a lesson well learned.

Displaying Magento tier prices instead of main prices for groups

I've been tackling a particularly annoying problem with Magento, in that I've got specific pricing by groups, imported from another database, but while it'd show the tiered prices in a box, it'd always show RRP on the front page. Fixing it to show the first tier price for quantity 1 on the product page was easy enough (using $this->getTierPrices()), but the tier prices simply aren't present on the grid / listing page, and it really didn't want to let me retrieve them by group either.

So, in case it helps anyone else out at a later date, this is how I got around it, displaying tier prices instead of the main prices on the listing page as well as the main product display page. (Tested on Magento 1.4.1.1, later versions may vary.)

Around line 58, after the following

getPrice($_product, $_product->getPrice()) ?>
getPrice($_product, $_product->getPrice(), $_simplePricesTax) ?>
getPrice($_product, $_product->getFinalPrice()) ?>
getPrice($_product, $_product->getFinalPrice(), true) ?>

add:
getTierPrices();
if (empty($_tierPrices) && $this->helper('customer')->isLoggedIn()) { // No tier prices found, attempt manual retrieval if customer logged in
$resource = Mage::getSingleton('core/resource');
$query = 'SELECT qty AS price_qty,value AS price FROM '.$resource->getTableName('catalog/product').'_tier_price WHERE entity_id = '.$_id.' AND qty = 1 AND customer_group_id = '.Mage::getSingleton('customer/session')->getCustomerGroupId();
$_tierPrices = $resource->getConnection('core_read')->fetchAll($query);
}
if (!empty($_tierPrices)) {
foreach($_tierPrices AS $tierPrice) {
if ($tierPrice['price_qty'] != 1) continue;
$_price = $tierPrice['price'];
$_finalPrice = $this->helper('tax')->getPrice($_product, $tierPrice['price']);
$_finalPriceInclTax = $this->helper('tax')->getPrice($_product, $tierPrice['price'], true);
break;
}
} ?>


I also had to alter the following line lower down to avoid displaying "as low as" for items that didn't have lower than the first tier price:

From:
getDisplayMinimalPrice() && $_minimalPriceValue && $_minimalPriceValue < $_product->getFinalPrice()): ?>

To:
getDisplayMinimalPrice() && $_minimalPriceValue && $_minimalPriceValue < $_finalPrice): ?>

Hope this helps someone out, as it had me scratching my head for a bit, and the online documentation I found wasn't brilliant. In the end the most efficient way seemed to be to rip it straight out of the database. Credit to the stack overflow question that pointed me in the right direction to run the query, modified to select only the customer logged in group and quantity 1 for efficiency.

Exciting adventures with ZeroShell and Hyper-V

Perhaps "exciting" was a little overenthusiastic, but nonetheless I'm happy to publish my rebuild of ZeroShell that runs under Microsoft Hyper-V, at least on our Windows 2008 R2 Core server. This was... an interesting journey, involving a lot of kernel recompilation, patching, initrd editing and banging my head against the desk, but with a bit of shoehorning, the synthetic network adapters and the virtual storage system seem to be working quite well. I've not put it under stress testing or long-term tests yet though.

I've posted up at the ZeroShell forums with the full information on it, and would be quite happy to hear any experience anyone has with it, or if it works for them. I'm hoping to use it to do bonded VPN over our ADSL lines here on a virtualised system rather than an old PC that is currently running pfSense. Porting the firewall rules in is going to be an interesting experience, although I suspect I'll use Firewall Builder on an Ubuntu VM on my PC here, but that's going to be a story for another day.

The direct download link is here. You may need to delete /Database/var/register/system/net/interfaces/ETH* to have your network interfaces appear "normally", as I didn't clean it up before compressing it - sorry about that! More details (and no doubt feedback) will also be on the link above for the ZeroShell forums.

Windows 8 Consumer Preview: First impressions

I got my hands on the Consumer Preview of Windows 8 courtesy of Microsoft, imaged my netbook and PC, and decided initially to try a clean install on the netbook (a Dell Mini 9, 2GB RAM, 32GB SSD, 32GB SD card, 3G modem built-in, wifi/bluetooth etc, 1024x600 widescreen display). All installed fine, until rapidly realising that I couldn't actually use most of the metro apps. Then noticed I was running in 800x600, and had no display adapter installed. Fine, installed the Windows 7 driver, all great and accelerated.

... And still couldn't run the metro apps. Minimum stipulated resolution is 1024x768.

I did ugly registry hacks so they would work, via a virtual resolution scaling tweak (1024x768 into 1024x600, or 1152x864->1024x600) and they worked fantastically, albeit they looked somewhat lousy and stretched. I can understand locking a minimum resolution for games, but for the messaging app and mail clients, and Metro-ified IE? Pointless, utterly pointless, they'd have fit perfectly well. It's an arbitrary setting. The suggestion I'm going to stick on the MS feedback forums, and I hope it doesn't fall on deaf ears, is that each application should have a minimum resolution setting internally or via the registry or similar, and if it's below the minimum acceptable for that application - and to be honest, 1024x600 would likely be the bottom end, as there's a lot of the older netbooks that are at that resolution. Fine, don't give me the pinball or the option to run the apps that require a bigger screen, but don't impose that I can't run any because it doesn't fit an arbitrary minimum resolution, when they clearly would perfectly well.

After this, I wiped the netbook again, and tried doing an upgrade from the preview, as it hadn't picked up my built-in 3G mobile broadband card properly (by which I mean "it didn't show the signal like it did in Windows 7, and I couldn't do the nice automatic setup"), and I couldn't find the bluetooth system anywhere. It got all the way to the end... then told me it'd failed, and rolled it back. Nice. Installed it clean again, had a play with it, but it just seemed to be like Windows 7, with extra bits I couldn't actually use. I got severely irritated, and imaged it back to Windows 7.

Not to be deterred, I also installed it on my main desktop PC, after duly taking an image. On here, I fared a bit better - the Metro apps seemed to work. Well, they launched, anyway, configuring quite a few of them was nothing short of a nightmare. Clicking on add account to mail or messaging brought up blank boxes, attempting to add email accounts to the Metro mail client often got an immediate response of being unable to add the account or to provide further details - yet when I rebooted it worked?! The messaging client was quite annoying too - it was actually nice enough to use, but refused to actually sign out. Using the new Metro start screen wasn't too bad. As soon as you assigned Chrome as your default browser, Metro IE wouldn't work anymore (it reverted to desktop), and the app store just plain didn't work most of the time for me once I tried to clean up the duplicate machines I had due to multiple reinstalls, no matter how I tried to re-add the machine to the app store. Apparently once it's gone, it's gone, and I wasn't going to reinstall or delete my user for a few Metro apps. It also felt very weird that I couldn't just drag the start screen across with my mouse and clicking/dragging, like I would on a touchscreen - at least, I'm assuming I would. It felt completely counter-intuitive to use a scroll wheel to have to move across. Another note on counter-intuitiveness - the "charms", as they're apparently called. Top right and bottom right of the screen pull up a bar for settings and the like on the right hand side... yet I found myself continually going to the right hand side of the screen to "request" said bar. That bar should really just appear when you're on the right, if you want it to make any sense. The top left task switcher seemed like "metro app, metro app, metro app, WINDOWS DESKTOP AND ALL IT'S APPS, metro app", and the start screen was... well. It didn't make me happy. Clicking start, typing in what I'm looking for - great! You got that right, Microsoft, good show. It's what I do on Windows 7. But did you really need to throw that interface in, or is it change for the sake of it? Another bug that annoyed me - when I was networking into my NAS box, it plain refused to remember my password, regardless of clicking the "Remember credentials" button. Not sure why, could have been because it was the same name (capitalised and all) from Windows with a different password? Who knows. I'll report it anyway.

There's a lot of good in there. It's fast. From reading up, looks like they threw out some of the old compatibility crud. This is good. If people still need stuff that's just compatible with XP, I've no doubt XP mode will be made available in the same way it was for 7, and the old support for the likes of Windows 95/98 apps really needed cleaning out a long time ago. There's some nice concepts in Metro, but they need major polish, and an interface should be complementing my thought process, not making me switch mental contexts when I want to do things, and not getting in my way. That I think is my biggest complaint - Metro felt in the way, not a complimentary technology, just unnecessary cruft thrown in my face because that's the direction Microsoft want to go. Mind you, if they gave me the 7 start menu on the 8 core, I'd be happy with that, too. I can't see that happening any time soon though, it seems to be far too big a part of their strategy, and if it's not different enough, they'll not see people buying it. Of course, they could go the OS X route - "Here's your upgrade, it's not that unlike the last version but it's better, faster, and you'll appreciate the changes, £30 please" - which I'd happily pay. But again, I can't see that happening, seems they're already quite set on their course, and that would no doubt make the bean counters quite unhappy, not getting the margin they're looking for, although I suspect they'd sell a whole lot more units. Still, mere speculation, of course.

Sorry Microsoft, but unless you do something to drastically improve the experience between the Consumer Preview and the final thing, it's not something I'll be jumping on, or recommending to anyone. Which is a shame, I had a lot of hope for this operating system, and a lot of enthusiasm. Let's hope you do a better job with Windows Phone 8 - I quite fancy one of those Nokia Pureview cameras on a multi-core Windows 8 Phone. Hopefully Windows 8 apps will make it easier to get Windows 8 Phone apps too, as it's currently trounced on that score by iOS and Android, although there's obviously a lot of pointless rubbish on said app stores.

I want something that'll do everything I want, and not get in my way doing it. Do that, and you've got my custom and my heart, on the PC desktop. The phone side too for that matter. Until then, I'll be sticking to Windows 7, which does everything I need it to rather admirably. I'm wondering if Microsoft have put something that good together in Windows 7 that they're going to face the XP effect, which is still on desktops 11 years later, because it was good enough for the purpose.

Choosing a phone...

As mentioned in my previous post, I have a love/hate relationship with my Nokia N900. It's great as a computer, but not so hot as a phone, and frankly I'm sick of having to hack stuff to death to get it working in the way I want. To be honest, that's been the basis for a lot of my change in philosophies lately - I shouldn't have to hack and squeeze and force things into place to make them happen, I want things to "just work", or "just work after a bit of poking into the right shape". I'm not averse to a little bit of configuration, but I want things to work the way I want afterwards, not having to cook my own ROMs or mess with bootloaders.

The truth is I'm fed up of hacking. I'm keeping the N900, naturally, as my pocket portable computing device. I'll have a portable wifi hotspot shortly that I can use for the purpose. But for my phone... I want something reliable that works, reads my work mail properly via Exchange ActiveSync, reads my personal email properly via a push mechanism, syncs up my contacts/calendars/etc, plays music well, has a reasonable battery life, has a reasonable storage space, a hardware keyboard would be nice but I'm not averse to using a decent on-screen one if it's available, reasonable audio/video streaming from the internet, high reliability, good resilience to day-to-day usage, hooks up to my preferred social networking sites, plays games for when I'm in the mood (particularly emulated ones are of interest - I'm a sucker for Sonic the Hedgehog on the Megadrive, used to love that, although I'm happy to get ported versions if applicable), a decent web browser, decent calendar, decent camera - I want a phone with a proper camera, sick of pictures that are just "okay" off mine - decent task manager... mind you, hell, I put a list together, on my laptop, which I was checking against an iPhone in the Apple store before the Apple shop police came along and asked me ever so politely if I'd mind putting it away while I was there as they were also trying to sell computers. I hadn't even considered using pen and paper, or anything quite so low-tech, and didn't have the dropboxed file on my phone at the time.

This is what I use my phone for:

1. Sleep analyser / sleep pattern alarm (not essential, but nice)
2. Alarm
3. Streaming internet radio
4. Games (sonic the hedgehog, emulated, Megadrive, substitutions allowed)
5. Playing music
6. Satellite navigation
7. Lookup nearest [x]
8. Price comparison of products
9. Occasional wifi hotspot
10. Update FB, twitter, G+
11. Use a calculator
12. Browse the web
13. Make notes - preferably in some sort of filed fashion, unlike my random notepad files on the phone right now
14. Find contacts, browse history of communication with them (via IM/text/email/whatever) - my current phone isn't brilliant at merging the histories, but the above would be nice.
15. TV remote
16. Windows remote desktop
17. YouTube
18. Stopwatch
19. Cooking timer
20. Bandwidth monitoring
21. Logitech Squeezebox control
22. Parking car - or more precisely, finding where I parked it
23. Taking pictures
24. Taking video
25. Playing music on headphones/speakers via 3.5mm jack
26. Send/receive files via Bluetooth
27. Stream Bluetooth audio
28. Occasionally: FM receiver
29. Occasionally: FM transmitter

Would like:

1. Augmented reality stuff, like the AR satnav, and other cool apps. Purely because it's cool.
2. Ability to stream media to other devices. Basic on-device DLNA/UPnP type server would do.
3. Properly combined contact activity streams - including G+ if possible.
4. Micro-USB charging, as everything else is also moving towards using it.

I know that "TV remote" is likely a no-go on any modern phone due to the lack of infrared ports, so I'm not bothered about that. FM receiver isn't used much, as the quality tends to be quite dire. The transmitter tends to be if I'm in someone else's car, but again, it's pretty rare, and it's not like there isn't accessories about to do it - I've a standalone one floating around somewhere myself anyway, so again, non-essential. No 3.5mm jack is a dealbreaker. I also don't want to have to pay money for extra things just to be able to do tasks I can already quite happily do, if possible. I understand a small extra outlay may be required for some things, and losing the massive 32GB storage on the N900 (I say that, but I'm using about 45GB of videos, music and software on there from memory, via internal and SD card!) would hurt unless I get something roughly equivalent, but the above is my thoughts.

Since then, I've done trawling around the Trafford Centre, and worked out that visiting phone shops is, for the most part, pointless. One thing I do know - between trying to use it between cable ties - is that my initial impressions of the Nokia Lumia 800 weren't good. The browser didn't seem intuitive at all, I couldn't figure out initially how to navigate anywhere, and things seemed to keep randomly going between applications - but buttons may have been being pressed by the cable ties. One of the guys in the shop did recommend against going anywhere near the Lumia though, as apparently they've "had nothing but problems with it, and keep getting them brought back" - his suggestion was the Galaxy S-II or the HTC Sensation XL, both of which seemed reasonably good phones, and the Galaxy S-II had a nice bright display with the AMOLED kit. Again, didn't really get much of a play with them, except that they had the fairly generic Android experience (well, HTC Sense on the HTC, and whatever the Samsung front-end is). Not much of a fair trial though. I also managed to get shouted at in the Apple store for having my netbook out and making notes on it on the counter while attempting to try out an iPhone 4S - only to discover it was crippled beyond belief, and you got what you were looking at. It did seem to scream the whole Apple philosophy - "you can do what you want as long as it's what we want" - although I suppose I can understand them not wanting to see competitor's tech in there (the wording, which was asked ever so nicely, was "would you mind putting that away, as we're also selling computers in here." He was ever so nice about it, but it completely took me off guard, I hadn't even considered any other method of taking notes). Yet they weren't bothered about my phone. *shrug* Did kill off any interest I had of being in there from that moment on though. However, the overwhelming conclusion of that was that apparently people want to look at phones in phone shops and go "ooh, that's shiny" or "Oh, that comes in pink, I like that!", rather than actually properly road-test phones. Perhaps I should have made an appointment like I was planning on doing in order to properly test one out from scratch - but I may still be able to with a few phones, more to follow on that one. I'm not allowing my phone-shop experience to influence my end decision, as, in all honesty, the only real thing I experienced was an in-shop Lumia playing up.

So far the shortlist looks a bit like this:

• The Lumia 800 (or any Windows Phone around the same sort of spec, really - annoyingly all of which come without hardware keyboards, unless I go back a generation to the likes of the HTC 7 Pro) - which also has the advantage like all Windows Phones that I'll be able to throw code at it once I'm up to speed with C#
• An iPhone 4S - until I properly try one, I've no idea if it's what I'd actually want, no chance of getting code on it though unless I decide to learn Objective C or involve the likes of MonoTouch
• A high-end Android phone e.g. the Galaxy S-II (again, same problem with lack of keyboards, although the previous-generation spec Sony Xperia Pro is a reasonable contender, albeit far from cutting edge... but it is also £200, a third to a quarter of the costs of the above), and again, no code without the likes of Mono for Android

I did also consider some sort of barely-above-dumbphone and a tablet, or similar, but it'd probably just annoy me quite quickly.

So. Next, I need to road-test the above, and work out how they'll fit into my digital life. The one thing whichever it is needs to be able to make me happy, not irritated, and be a lubricant in the chafing of life, rather than something abrasive, which I have to fight every step of the way to achieve what I want. I'm tired of fighting with technology, it should be working for me, not the other way around.