Saturday, December 1, 2012

DUAL BOOT with PREINSTALLED WINDOWS 8 and LINUX


Step 1: Introduction

This is simply an intro. If you want to save time, skip reading here, and go directly to part 2. There is no vital technical information here.
I've been looking for at small, cheap computer to play the role as a flexible, small (and smart) TTY/SSH client for a while. I was looking for a used 10.1” mini-PC, expecting to install OpenSUSE / CentOS on it. Suddenly, I stumbled over a dirt cheap, brand new, 64bit 17” computer, Windows 8 preinstalled – the works!

It was an HP DV6 Pavillion. Learn W8, the computer, install and adapt SW – and investigating how to keep Win8 while dual-booting into CentOS 6.X, OpenSUSE 12.3. After all, this was all new to me: W8, OpenSUSE 12.3, CENT OS, UBUNTU, UEFI, Secure Boot, GPT.

While investigating this, I found that many others were wondering about the same, and I did not find one posting that answered all my questions. So I decided to publish my findings when I had something to report. It seems to have some kind of “magic” connected to it to get things working, and that the results may be unreliable.

I set out to assemble knowledge in order to sort it all out, and to find a way to do it reliably. What is below is the results of my findings. I've tried to make it systematic and easy to read, yet complete. Where I think it isn't complete, I've said so. If you think I've missed something, please say so. There isn't much that is my own thoughts – most of it has been derived by piecing together information found elsewhere on the Internet.

My impression and bold standing on this issue, is that as of now, dual-booting a preinstalled Windows 8 through UEFI Secure Boot is working, and behaves predictable - provided you do the right things, and that you do it right (isn't it always so?). The “only” unknown factor remaining, it seems, is what is being introduced by the firmware vendors – and that is inevitably going to improve as time goes by as well. So – if you've got a fairly new BIOS (which you probably do have, since you have Windows 8 preinstalled), the chances are that you will be fine.

I'll split everything into what I believe are logical parts to make it easy to look up what you need. Except for the troubleshooting part, which you hopefully will not need (I didn't – at least up to now), you'll need to broadly follow the sequence I've listed, though – at least on the “installment” level.

As you will see, I am not fully satisfied with my findings yet, as I want more tools and procedures to aid in the case of problems, although I did not run into any problems when doing as described below. However, please do supply any info you have to add, and be sure to complement, complete or correct any errors of mine. I will continue to investigate the troubleshooting part for some time – and update this thread when/if I find something. If I don't, I'll wait for some more time to go by – then return and update when something interesting has arrived. However, it is inevitable that the relevance of this thread will lessen as the UEFI/Dual-boot scenario matures and better support is offered from everywhere. My bet is that with the next release of OpenSUSE, most (all?) new PCs shipping will be reliable as far as UEFI/DualBoot goes.

Step 2: Getting to know the terminology (BUZZ Words):

First of all: Here are some links I found useful and that can be looked up by those who want more details. You'll find that many of these links point to each other, as they can be seen as references in its own right:

  1. OpenSUSE: OpenSUSE's own UEFI de-mystifier, simple and technically advanced at the same time. Truly a good starting point for further study.
  2. Booting from GPT: Rod Smith's excellent site about everything UEFI/GPT. This link enters somewhere "in the middle" of his site, but holds links to the other parts as well. Truly useful. His tool GPTFdisk (gdisk) is part of the OpenSUSE 12.3 distribution.
  3. EFI System partition: About ESP on Wikipedia.
  4. GRUB/GRUB2:
  5. GUID Partition Table:
Some links from the Microsoft site. Since W8 may disturb dualbooting in its own favour, one can be helped by fixing things from the W8 side. Most of these evolve around Microsoft's BCDEdit.exe, which is a tool that seems to be able to handle almost everything in the UEFI partition, but is terribly documented. "Terribly" in this case means it seems to be documented, but it is so scattered and cluttered, that it really takes time and effort to put it all together – and to get a clear overview of what the terms Microsoft is using really means in practical life. I haven't done that yet, and it will have to wait until I run into a problem I can't resolve in any other way, I am assigned a task that makes that sensible, or a rainy day when I lack other things to do.
  1. Windows and GPT FAQ: with few dos and don'ts.
  2. Windows 8 boot options: Overview of Boot Options in Windows Vista and Later (Windows Drivers).
  3. BCD Boot options reference: 
  4. Editing boot options (Windows Driver)
  5. Using boot parameter (Windows Driver)
  6. BCDboot command line options.
  7. BCDEdit.exe command line options:
What is GPT?
GPT is short for GUID (or UUID) PT which is short for Globally Unique ID Partition Table. This is replacing the old MBR (Master Boot Record) partitioning scheme which we are all accustomed to. This is inevitable due to the limitations of MBR. In practice and for the short term/immediate concequences, the most visible change is that GPT's partitions are what is known in MBR as primary partitions, and that there can be (many) more than 4. (In fact, 128 partitions for a start – you can extend that). I.e my Asus S200E came with 6 primary partitions. The ESP partition is one of these, but this partition is not required on a GPT partitioned disk. On Windows GPT prepared disks, you will also find a small (mostly 128MB in size) MSR (=Microsoft Reserved) partition. As with MBR, only one GPT partition can be the System partition (the one that control is transferred to during the boot-process). In an UEFI-based system, this must be the ESP partition you want to control the proceedings.

What is UEFI?

What is the MSR partition?
MSR (Microsoft Reserved partition) is a Microsoft specific, small, hidden (from within the OSes, but GPT does not allow for hidden partitions in the way MBR did), unformatted partition Microsoft require to be present on all its GPT partitioned disks. The size is 32MB for disk sizes below 16GB, 128MB for the bigger disks. Provided I haven't misunderstood MS, MS (and probably also third-party SW components with the right requirement) will "erode" this partition into more partitions; i.e. if you repartition your drive in certain ways ("Dynamic disks"), thus adding more partitions to your drive. For the time being, there is a limited number of partitioners that can handle this partition. gdisk (included in OpenSUSE 12.3) is one, and Microsoft's DiskPart.exe is an obvious alternative. Beware though, both of these have a different user interface from fdisk, so there is some "getting used to" involved. PartEd Magic's live CD (2013-05-01 edition) will display and delete the MSR partition, but nothing else. So, if you want GPT partitioning and want to access these from both Windows and Linux, be sure to create an MSR partition as well when you first partition it.
See Microsoft's GPT FAQ (see link above) if you want more details.

What is the ESP partition?
In an UEFI system, ESP (=EFI System Partition) is the partition that holds the boot-loaders that in turn loads the OS you want. The UEFI specification does not limit the number of ESP partitions on a disk, nor does it put restrictions on their positions, but only one can be marked System in the GPT at one time. The one marked System, will be the one controlling the boot process. The ESP partitions are FAT formatted (Windows 7 needs FAT32, my Windows 8 preinstallation file set also came with FAT32 formatted ESP partition for Windows 8 – I haven't tried changing that).

The ESP partition: How many are needed, where, and how big should it be?
Different sources on the internet says different things. Microsft only support one such partition – and that partition needs to be on your boot disk. MS also recommends it to be the first partition on your disk. As for size, according to Rod Smith (see link above) Windows installers create 100MB ESP partitions, Macs create 200MB, while Linux utilities gets confused with FAT16/32 setups if partitions are smaller than 520MiB, so he recommends creating 550MiB for the ESP. My system came with a 300MB ESP partition.

A comment of mine on the 128 possible GPT partitions:
As this lifts the pressure from the 4 MBR partitions we have been struggling with for years, I look forward to see how this evolves. The immediate effect is obvious. However, resources available will be used, and I expect to see new and innovative ways of exploiting the now "endless" resource that the number of partitions is. Just see how fast we passed the 16MB limit of the IBM PCAT when we at last passed the 640KB memory limit of Intel Real Mode in IBM PCs. Microsoft's way of using the MSR-partition gives us some pointers: We can expect to see many small partitions for special use. Maybe 128 partitions will end up to be a hurdle after all? No problem – we can just expand that, 128 partitions isn't a limitation of the specification. Doing so, however, does include repartitioning of your disk – and Microsoft says "Windows implementation restricts this to 128 partitions" (see GPT FAQ in link above). So – in a dual-booting Windows/Linux system, 128 partitions may become a serious limitation sooner than we think – simply because nobody sees it as a problem the way 4 MBR partitions (almost) always were, and start creating partitions at a rate we've never seen before.

The BIOS/UEFI Firmware problem:
This is the one problem you and I cannot do much about apart from making the right choice when purchasing the PC. We are stuck with what the vendor provides. My reading on the 'net indicate that there still are some differences here and there, and that some of them will not do in a dual boot Windows 8/Linux setup. 
However, a computer that comes with Windows 8 factory installed, is certified (by Microsoft) to conform with UEFI-standards and secure boot. That should help. Also, OpenSUSE 12.3 64-bit edition do support UEFI-booting, secure-boot incuded.
Make sure, however, that you do upgrade your BIOS to the latest one published – even if you have a brand-new PC! Upgrading (anything) prior to properly starting to use it, has helped me many times over the years. By doing that, I have avoided many problems others have been banging their heads into.
In my case, the computer I bought Tuesday last week, seems to be manufactured in December 2012 (if I read the labels correctly). The latest BIOS (version 202) published, was dated October 26th 2012. I was surprised to see that my BIOS was version 203! As I am writing this, I see that more have happended on Asus' support site: In the period of the last few days, they have been publishing several newer version of my BIOS, the latest one is dated yesterday – version 209. I still have work to do ...
The bottom line is: If you purchase a PC that comes with Windows 8 factory installed, chances are good that your BIOS is up to the task of dualbooting into Linux. Other HW drivers however, is the same game as it has always been.

Secure boot: What is it, do I need it and what exactly does it provide?
Non-technically spoken, and a little inaccurate, this is a way to make sure that no one can modify the files involved in loading an OS without it being detected. If such modification is detected, booting will be interrupted. Such a mechanism makes it more difficult to sneak in viruses, root-kits etc. However, if you do run into problems (or you for some reason don't want it), you can disable secure booting. That will enable booting without the modification-checks, and the OS will be loaded. To do its job, secure-boot involves X509 certificates, encryption and digital signatures, and protects OS-files until the kernel has been read and loaded. This is equal for Windows and OpenSUSE 12.3. If you want more details, see OpenSUSE's UEFI demystifier link and the link to Rod Smith's site above.

Getting to know Windows 8:
I know, this is a Linux site, but bear with me. What is listed here isn't much, and it makes for some shorter text elsewhere in this thread – and it is tidy to keep it in a collection. <Wkey> means the Windows-key present on all Windows PC-style keyboards:<Wkey>-X. If you are like me, this menu will soon be your favourite Windows 8 menu
  • <Wkey>-Q All applications - "The Metro UI"
  • <Wkey>-I Settings shortcut + power off and some more.
  • <Wkey>-C Same as moving mouse-pointer to upper right corner of the screen.
  • <Wkey>-E Start Windows explorer (File Manager).
  • <Wkey>-D Access the Desktop, as close to the old-fashioned desktop style as you gets in W8 for now (without 3rd-party tools).
  • <Wkey> Toggle between the Metro GUI's Start screen and Windows 8 Desktop.
  • <Alt>-F4 still works as you would expect
  • <Alt>-<Tab> (and <Alt>-<Shift>-<Tab>) also work as you expect.
Windows 8.1 is rumored to give you back the Start button...

Step 3: Decisions

Based on the info given in Part 2 above, I made my decisions about how to set up my computer: You should make similar decisions for yourself.
  1. Number of ESP partitions: Due to Microsoft's recommendations in its GPT FAQ (see link above), I am reluctant to have more than one ESP on a disk in a system that will coexist with Windows. However, due to other comments I've read (I am not able to find it back, sorry) making an ESP partition mandatory for any disk may come in handy for Linux type of systems (and Microsoft doesn't advice against it either) so I will put an ESP as the first partition (and an MSR partition as the second partition if it is to be used in combination with Windows) on any data disk i set up with GPT.
  2. The size of the ESP: In spite of Rod Smith's advice, I am going to keep my ESP partition at 300MB. If, however, I were to do partition my disk from the start using GPT, I probably would have followed his advise and set it to 550MiB.
  3. The other partitions: My system came with 6 partitions on its 320GB disk:
    1. 300MB EFI System Partition FAT32
    2. 900MB Recovery Partition NTFS (this is an OEM partition)
    3. 128MB MSR Partition Required. No data, not formatted.
    4. 119GB OS (my W8 drive C: NTFS)
    5. 157GB Data (my W8 drive D: NTFS)
    6. 20GB Recovery Partition NTFS
  • I decided to simply delete partition 5 and 6, and make them available to OpenSUSE. I also tried to delete partition 2 (I could) and move partition 3 onto partition 1 (I couldn't – at least, I didn't find out how without much hazzle) so I kept partition 2 all the same. Not much space wasted anyway. (Don't remove any partitions of yours yet).
  • For OpenSUSE, I decided for 4GiB + 128MB as swap size. As I read somewhere (I have yet to test it) that Linux uses the swap partition for hibernation, the swap partition needs to fit the amount of RAM + a little more. I have 4GB of RAM on my system, and 128MB is a little more.
  • The remaining space I split equally between the root (/) and /home partitions, as I do not yet have a clear picture as of where I need space the most. But I sure want an easy upgrade path to the next release, so I wanted a proper /home partition to keep.

Step 4: Preparation

Preparing your computer for dual booting:

  • You should do a full backup of your harddisk. I used CloneZilla live-CD and saved a full disk-copy onto an external USB-disk prior to booting any OS (Windows 8) the first time. Always download and use a new version of CloneZilla for a new PC. That's the best way to make sure that the technologies supported match the technologies needed. I had to switch off UEFI boot mode (launch CMS), and disable secure-boot to boot the CloneZilla live-CD. See below: “The steps I followed” to see what I did to go through it all.
  • Visit your computer vendor's support-page and check that you have the latest version of BIOS installed. If you do not, upgrade your BIOS. Make sure that you are verifying the right edition of your BIOS, and that you do not pick the wrong one. My computer had two BIOS editions, each with its own set of versions. Asus was very clear about how to pick the right BIOS edition.
  • Boot your PC into Windows 8 and let Windows install itself, if you have not already done so. Make adjustments to your liking.

Preparing Windows 8 for dual-booting:

  1. Do a complete upgrade of Windows 8 using Windows Update.
  2. Also, if your vendor publish any Windows-specific utilities for your PC, upgrade these too – particularly those being BIOS/HW-configuration oriented.
  3. Back up the EFS partition:
    1. <Wkey>-X --> "Command Prompt (Admin)". Confirm.
    2. Create (as you need) and go to a directory where you want to store the contents of the EFS partition.
    3. Enter the command: [$ bcdedit /export .\sysdefault.uefi] to save the contents of the EFS partition in the file sysdefault.uefi in the current directory. This may come in handy if a simple EFS adjustment is all that is needed later on...
  4. To keep Windows 8 from cluttering up the EFI NVRAM (and thereby hiding your soon-to-be important OS from you):
    1. <Wkey>-X --> Control Panel.
    2. Select category "All Control Panel Items"
    3. Select "Power Options"
    4. Click "Choose what the power buttons do"
    5. Click "Change settings that are currently unavailable"
    6. Scroll down to unveil "Shutdown settings"
    7. Make sure that two of these values are unchecked, namely:
      1. "Turn on fast start-up (recommended)" and
      2. "Hibernate"
      3. Uncheck any or both of these as required.
  5. If you did a full Windows 8 backup under "Preparing your computer..." above, you can skip this, but having this one will make it easier to fix problems without having to reset your computer all the way back to factory settings. So, I recommend you to go on and create a Windows 8 recovery drive:
    1. <Wkey>-X --> Control Panel.
    2. Select category "All Control Panel Items"
    3. Select "Recovery"
    4. Choose "Create a recovery drive"
      1. This demands a USB drive (memory stick – Windows will reformat it!), on my system minimum 16GB USB drive was required, I've read someone quoting 32GB being required. I suppose that depends on what the OEM preinstalls. My 32 GB USB drive had 19GB free (of 29GB total capacity) after performing this action, which reflects the contents of partition 6 on my system.
      2. I was offered to delete the recovery partition to free up some drive space. I did. I was going to anyway (see above). This resulted in another drive being made available to Windows 8: Drive G: totalling 20GB of additional free disk space.
  6. Power off Windows 8
    1. <Wkey>-I --> Power --> "Shut down"
This concludes Windows 8 preparations, and Windows 8 should now be ready to cooperate properly in a dual-boot UEFI Secure boot setup.

Some more preparations:

I chose to use the PartEd Magic live-CD once again for this (remember to download the latest version if you do too), but if you feel confident in OpenSUSE's setup partitioner, you can do just the same as shown here from there. If you do not feel fluent in OpenSUSE's disk partitioner, I recommend you to do as I did, and describes here:
  1. Boot using PartEd Magic (latest version - I used the 2013-05-01 edition).
    1. To do that, I had to connect a USB DVD-reader to boot, and needed to switch to BIOS/MBR boot emulation and disable secure boot:
      1. On my PC, that meant power on while holding down F2 until the BIOS setup screen appeared. Your PC may have other demands, e.g. Del (used to be standard on Dell computers), F10, F12 or something else.
      2. From the BIOS top menu, I chose "Boot", then "Launch CSM" and changed that from "Disabled" to "Enabled", Then I selected "Security" in the BIOS top menu, then "Secure Boot Control" and changed that from "Enabled" to "Disabled".
      3. I then pressed F10 to save and exit.
  2. Upon the reboot that followed, I had to wake up the boot-source selection menu. On my computer, this is done by pressing and holding the Esc-key until the boot-menu is displayed. If PartEd Magic is the only non-GPT partitioned, bootable medium available, your BIOS may boot PartEd Magic directly.
    1. I selected my USB DVD-drive and waited for the Parted Magic menu to be displayed.
  3. Since this is a 64-bit computer, and I am not native English, I selected item "2+", and selected my language from the next screen (again, the "64" alternative).
  4. READ THIS ITEM TO THE END BEFORE YOU DO ANYTHING! Start up the disk partitioner from the desktop, and delete partitions you do not want. If no partitions are available for deletion (your Windows 8 C:-partition may span the disk from wherever-it-starts and to the end of the disk, resize it until you have ample disk space available for OpenSUSE.
    1. NOTE: On my PC, the Windows 8 C:-drive's partition is labeled "OS". You should make sure that you know which partition is which, and you should normally NOT need to remove any partitions prior to (and including) the Windows 8 C:-drive's partition. If you are unsure, revert what you did in item 1 above to enable booting Windows 8 again, restart Windows and get yourself aquainted with your disk's layout. Also see "My decisions" item 3 above for a comparison/help at that. When you know your way around in your partitions, go back here and start over at item 1 just above.
    2. On my system (see "My decisions" in item 3 above) I removed data volumes D: and G:, gaining 177GB of free disk space for OpenSUSE to use.
  5. Reboot and revert the BIOS settings you did in item 1 just above, in order to re-enable UEFI/GPT/Secure Boot.
  6. Do a new reboot, and check that W8 is still able to boot. It should work.
  7. Power off Windows 8
    1. <Wkey>-I --> Power --> "Shut down"

PART 5 Installation:

Setting up OpenSUSE 12.3 with UEFI and Secure boot:
This is actually quite simple: Follow the instructions documented by OpenSUSE. After selecting to install from OpenSUSEs DVD image that I had burned to a DVD, here is what I did:
  1. I tried to boot off the USB DVD, and lured the boot-menu to the front by holding down the <Esc>-key. It may be different on your computer.
    1. I got a boot menu, but my USB DVD drive was not among the alternatives. So, I thought I had to migrate the DVD onto a USB-memory stick. However, there was a "Asus UEFI boot"-alternative available. By selecting that, the USB DVD-drive started spinning, and the OpenSUSE 12.3 installation menu was displayed.
  2. I hit <enter> for "Installation" and waited until the Welcome-screen was displayed. There I made my choice of language and keyboard and clicked <next>.
  3. I arrived at the Installation Mode -screen. I selected New Installation and checked kept the prefilled check-boxes (unchecked: Include Add-on Products from Separate Media, checked: Use Automatic Configuration). <Next>.
  4. Clock and Time Zone-screen. I made choices relevant for me by selecting country/time zone on the map. I kept the "Hardware Clock Set to UTC"-check-box unchecked. <Next>
  5. Desktop Selection. KDE. <Next>
  6. Suggested Partitioning:
    1. As I already had 4 partitions left after removing 2 (sd1 through sda4), the first partition available for OpenSUSE would be named sd5. Here, OpenSUSE suggested a scheme that would make use of the free area on my HDD: A swap volume as sda5 (2GB), a root volume as sda6/ext4 (20GB) and a home partition as sda7/ext4 (155GB). The EFI-part was set to: "Set mount point of /dev/sda1 to /boot/efi". That is a perfectly OK, scheme, and I could have accepted that.
      1. However: I want to try out OpenSUSE as a laptop-OS. I've read somewhere that Linux hibernates to the swap volume (I haven't tested that yet, but I intend to). That means swap needs to be a little bigger than the amount of RAM. So, having (only) 4GB RAM, I allocated 4GiB + 128MiB swap volume (=4224MiB).
      2. I also want to start experimenting with compiling stuff - kernels included - which I have experienced on earlier attempts, that sometimes require more space than immediately meets the eye.
      3. Having a separate home volume partition to be prepared for future upgrades makes perfectly good sence.
      4. To make it easy, I split the remaining disk space in two equal halves: 86,7GB each for root (/) and /home partitions.
      5. Thus, I selected "Edit Partition Setup" to that effect and made my adjustments. After finishing my partition-selections, I clicked "Accept".
      6. Then the OpenSUSE installation told me "You chose to install onto an existing partition that will not be formatted. Yast cannot guarantee your installation will succeed, particularly in any of the following cases...". Well. I double checked, and checked again. I had set up all my three, new partitions to be formatted. So, knowing Yast wasn't correct (could it be that ie. the MSR partition triggering this? That one is not formatted.) I answered Yes to Yast's challenging question: "Really keep the partition unformatted?"
  7. Returning to "Suggested Partitioning screen" again
    1. I checked that the partition info did reflect my selections and adjustments. It did. <Next>
  8. Create New User screen: I filled in the vaues as appropriate for me. (I always make the root and user passwords different, and I never use automatic login). < Next>
  9. Finally, I got to the "Installation Settings" screen. Here's the important UEFI part.
    1. I found the "Booting options" portion: Open SUSE did detect the UEFI correct, and correctly selected the GRUB2-EFI option. However, just as the Open SUSE documentation says, it didn't detect Secure Boot. I need that.
    2. I clicked the "Booting" choice.
    3. Boot loader Settings screen. I checked the "Enable Secure Boot Support" check-box. <OK>
    4. Back to the Installation Settings screen again, and read that my secure-boot choice was accepted.
  10. I made my choices for the other options, according to what (I think) I need, and want to experiment with at first. This is no different from any Open SUSE installation, as far as I know. (To be honest, I wandered through the options in detailed layout, but this time I didn't change anything apart from what I have mentioned above! I'm planning to replicate the 12.3 repository to an external drive later, so I can do what I like later without even being online - summer holidays are closing in ...) <Install>
  11. After a while (30+ minutes), the installation finished, and the PC rebooted. I didn't remove the DVD disk (as I didn't receive any instructions about doing so).
  12. After booting (from the HDD), OpenSUSE had automatically pre-selected OpenSUSE for me (I did see Windows 8 there as well!), the installation process continued with automatic configuration - still accessing the DVD during that process. Soon, that came to an end, and I could log in to Open SUSE for the first time - on my HDD, UEFI - Secure Boot - Dual Boot with Windows 8 and all... SUCCESS!

Step 6 Closing adjustments

Adjusting Windows 8 after everything else has been done:
If you boot into Windows 8 after carrying out all the entries above – and start Windows Explorer (e.g. using <Wkey>-E), you will find that the newly created Linux volumes are visible and (almost) ready to be used from Windows 8. They even have their own drive letters, even though they are marked as RAW, and having “no content”! If you click on any one of these drives, Windows 8 will rush to your assistance and offer you to format them, in order to make them useful! We do not want that! These Linux volumes needs to be hidden from use in Windows. You can do that in two ways; through the GUI or command line:
  1. The GUI-method: Press <Wkey>-X and select Disk Management. For each of the (“RAW”) Linux volumes, right-click on it and select “Change Drive Letter”. In the resulting Dialog box, you select the “Remove”-button in order to remove the drive-letter assignment. Confirm that you understand the consequences of the warning being displayed. Click OK. This effectively hides the processed volumes from Windows 8's Explorer.
  2. The Command-line method: Press <Wkey>-X and select “Console – Administrator mode”. Confirm. Enter the following command: mountvol <drive> /D and press the <Enter>-key. <drive> is to be replaced with the drive letters shown for each of the Linux volumes we are to hide from Windows 8, so you need to execute the command once per Linux volume.
An afterthought: When using gdisk under OpenSUSE, I saw that the GUID code used for the Linux partitions I created was the code used for Windows Basic Disk. That being so, it is only natural that Windows 8 displays those partitions as a standard data disk, and offering them to the user. These GUID codes were automatically selected/decided by the OpenSUSE 12.3 partitioner during OpenSUSE installation. I haven't read the GPT specifications, so this may be very reasonable (it may even have to do with secure-boot having to use certificates signed by Microsoft), but – having 64 bits at disposal for classing devices – there should be possible to come up with e.g. a Linux Basic Disk and eliminating this last adjustment to Windows 8, and thereby better protect Linux volumes from Windows' ignorance of other file systems. If this effect is due to OpenSUSE using Microsoft certificates, I am sure that this will change in the future when secure-boot is no longer experimental.
Securing the boot-setup: Repeat the step executed in Step # 4, Preparing Windows 8 for dual booting, item 3, but use a different file name that you did then. You should not overwrite the old file. This will save the modified EFS partition complete with what OpenSUSE added for it (and GRUB2) to do its magic.

Step 7 Troubleshooting:

This is the part that I am least content with. I welcome any contributions here.

There are several fields where you can get problems. For the "I am able to boot, but have problems within Linux (or Windows)" type of problems – and that is not directly related to disk partions/volumes, they are no different than they ever were in either OS, and I have to point you to the normal support channels for the OS in question.

If you get problems with booting or dual-booting, then that is what I aim to cover here. However, so far there isn't many tools available for us to use. And most of the tools that do exist, are fairly technically oriented and not meant for the occasional user. I haven't had any problems yet, so I must guess a little at what kind of problems one can run into, and I lack experience with the tools that do exist, so I cannot offer much details either. I will, however, fill in advice I receive from you all, if you are so kind as to supply me with them.
  1. Nothing is booting any more. This is the worst scenario.
    1. You can try booting from the Windows 8 recovery stick created in part 4 above, interrupt the process and go to the command-line. We will restore the ESP partition data we saved in part 6, Securing the boot-setup. Execute "bcdedit /import <file>" where <file> is the name of the file you used in part 6 above.
    2. If 1.1. above fails, the ESP partition itself is probably destroyed. That can be fixed in several ways. The easiest is probably to recreate it (e.g. using PartEd Magic live-CD), then restore it using the procedure of a) above.
    3. If 1.1. succeeds, but still no boot, I would find me an emergency boot medium (PartEd Magic or the Windows 8 recovery stick), and process the partitions using some disk-analyzing tool. Since Windows is what Windows is, with Windows based tools (at least the original Microsoft tools) you run the risk that the tools will destroy the contents of Linux partition (remember – Windows sees them as unformatted Windows Basic Disk volumes). Linux based tools are therfore my clear favourite here, but I know of no such tools yet, that isn't data destructive too. Sorry. (<Help anyone?>  )
  2. Only OpenSUSE will boot. Frankly, I think this is the least probable alternative.
    1. Look into what efibootmgr (included with OpenSUSE 12.3) can do for you. The link to OpenSUSE's EFI demystifier above, also provides an example of how to use efibootmgr. Try setting –bootnext and point it to Windows 8. If the problem is UEFI NVRAM only, that may rectify it.
    2. Is it only with secure-boot? OpenSUSE documentation says: "Secure boot may stop working in the future. If that happens, simply disable secure-boot – then commence OpenSUSE booting – and do an online upgrade. This will download a new version of shim that will work. After doing the maintenance, re-enable secure-boot and you are up to speed again."
  3. Only Windows 8 is booting – nothing to be seen of OpenSUSE: This is probably the more likely situation because we have to "tame" Windows 8 so it behaves itself in a dual-boot environment. This is probably an NVRAM issue, and can be remedied by using bcdedit and make it set OpenSUSE to "bootnext".
Tools of the trade: As I said, there are not much available, but there are some:
Windows based tools:
  • Microsoft's bcdedit.exe. (Included in Windows 8) Command-line based. It seems to be able to do nearly whatever we would ever want to do with an ESP partition, but terribly documented. Hard to understand because of that.
  • EasyBCD. (EasyBCD - NeoSmart Technologies) GUI based. Should be easy to use for what it can do, but has limited capability when compared to what the other tools have. Its community version is free for private and some non-profit use, but it is reported to silently insert commercial in other components. I haven't used it.
  • Visual BCD (http://www.boyans.net/). Makes use of the Windows WMI interface. Promising. Easy to use, and potentially far more powerful than EasyBCD. Free for personal use. However, the site lacks dates in general, and the license dates suggests there have been no development since 2011. UEFI isn't fully implemented, but is claimed to work. I receive some errors when I tried using it but nothing wrong has been detected. The uninstaller doesn't properly clean up the registry. When saving off the ESP partition, it produces a file of the same size as bcdedit.exe, but their contents are different.
Linux based tools: There is, of course, efibootmgr, (included in OpenSUSE 12.3) and that is all I have on the list for now. I am sure there are more. Please help me compile a proper list. I'll return later myself and fill in more, but I thought it was better to publish this article now and not delay all the other stuff.
Also, if a data disk is the problem: Windows 8 cannot read a newly partitioned GPT disk. Linux can. It has NTFS file system. Did you create the MSR partition?

Step 8 YAHOO!!!! Working for me :)

First impressions:
Everything seems to work well, apart from two things:
  1. I have not been able to find a HW driver for my 100Mbit cable-ethernet card. Since WiFi worked straight off the bat and I have an old USB/ethernet card lying around for service purposes that Linux do provide drivers for, I probably will not bother with it.
  2. The touchpad is misinterpreted and I cannot adjust it – not even in the fields displayed as being editable. I have a clickpad (not two separate buttons separated from the touchpad). However, I am able to use it in ways that shouldn't be possible if I understand the touchpad setup-windows correctly. Obviously, I am missing something. Oh well. I have to dig into that later.
The touch screen could be used to move Windows around when I was using the PartedMagic live-CD, but nothing else. In OpenSUSE, however, I can mark areas and click, drag and move objects as I wish, using the touch screen. However, since the GUI is still mouse/keyboard-oriented (and added to that, I currently have the LapTop scheme active), fields are pretty narrow for my fingers when using the touch-screen for it to be productive. 
However, much to my surprise, I see myself mixing the use of keyboard, touchpad and touch-screen (and mouse, when I have one connected) when interacting with the computer. I never thought it would be that tempting (and natural) to use the touch screen in that way. I thought the ergonomics of that would be terrible. It isn't. (At least not on a laptop computer, where I keep the screen considerably lower and closer to my body than when I use stationary monitors on table tops). You just get another method for interaction, and you use it as you see fit. My guess is that some will like it, and some will not.

Well. That was it. I hope someone will find some of it useful. Please feel free to comment and correct! Bye, now!

Friday, November 9, 2012

Linux / UNIX setup SSH with DSA public key authentication (Password less login)

Q. How do you set-up SSH with DSA public key authentication? I have Linux laptop called tom and remote Linux server called jerry. How do I setup DSA based authentication so I don’t have to type password?
Ans. DSA public key authentication can only be established on a per system / user basis only i.e. it is not system wide. You will be setting up ssh with DSA public key authentication for SSH version 2 on two machines:
#1 machine : your laptop called tom
#2 machine : your remote server called jerry

Command to type on your laptop/desktop (local computer)
First login to local computer called tom and type the following command.
Step #1: Generate DSA Key PairUse ssh-keygen command as follows:
$ ssh-keygen -t dsa
Output:
Enter file in which to save the key (/home/vivek/.ssh/id_dsa):  Press [Enter] key
Enter passphrase (empty for no passphrase): myPassword
Enter same passphrase again: myPassword
Your identification has been saved in /home/vivek/.ssh/id_dsa.
Your public key has been saved in /home/vivek/.ssh/id_dsa.pub.
The key fingerprint is:
04:be:15:ca:1d:0a:1e:e2:a7:e5:de:98:4f:b1:a6:01 vivek@vivek-desktop

Caution: a) Please enter a passphrase different from your account password and confirm the same.
b) The public key is written to /home/you/.ssh/id_dsa.pub.
c) The private key is written to /home/you/.ssh/id_dsa.
d) It is important you never-ever give out your private key.
Step #2: Set directory permissionNext make sure you have correct permission on .ssh directory:
$ cd
$ chmod 755 .ssh

Step #3: Copy public keyNow copy file ~/.ssh/id_dsa.pub on Machine #1 (tom) to remote server jerry as ~/.ssh/authorized_keys:
$ scp ~/.ssh/id_dsa.pub user@jerry:.ssh/authorized_keys
Command to type on your remote server called jerryLogin to your remote server and make sure permissions are set correct:
$ chmod 600 ~/.ssh/authorized_keys
Task: How do I login from client to server with DSA key?Use scp or ssh as follows from your local computer:
$ ssh user@jerry
$ ssh user@remote-server.com
$ scp file user@jerry:/tmp
You will still be asked for the passphrase for the DSA key file each time you connect to remote server called jerry, unless you either did not enter a passphrase when generating the DSA key pair.
Task: How do I login from client to server with DSA key but without typing a passhrase i.e. password-less login?
Type the following command at shell prompt:
$ exec /usr/bin/ssh-agent $SHELL
$ ssh-add

Output:
Enter passphrase for /home/vivek/.ssh/id_dsa: myPassword
Identity added: /home/vivek/.ssh/id_dsa (/home/vivek/.ssh/id_dsa)

Type your passhrase once. Now, you should not be prompted for a password whenever you use ssh, scp, or sftp command.
If you are using GUI such as Gnome use the command:
$ ssh-askpass
OR
$ /usr/lib/openssh/gnome-ssh-askpass
To save your passphrase during your GNOME session under Debian / Ubuntu, do as follows:
a) Click on System
b) Select Preferences
c) Select Session
d) Click on New
e) Enter "OpenSSH Password Management" in the Name text area
f) Enter /usr/lib/openssh/gnome-ssh-askpass in the command text area.

g) Click on close to save the changes
h) Log out and then log back into GNOME. After GNOME is started, a dialog box will appear prompting you for your passphrase. Enter the passphrase requested. From this point on, you should not be prompted for a password by ssh, scp, or sftp.

Sunday, October 14, 2012

Benefits of omeda-3 - Flaxseed

 I have discussed at length in previous articles the benefits of omega-3 fish oils. I have not, however, given you much information about flaxseed, another popular and very beneficial omega-3 fatty acid. Let's explore why we need it, and compare it to the omega-3 fatty acids in fish oil.

"Wherever flaxseed becomes a regular food item among the people, there will be better health," said Mahatma Gandhi.

He was right. The true nature of flax as a health food has been known for centuries. In the eighth century, Charlemagne considered flax so essential for health that he actually passed laws requiring its use. Flax was one of the original medicines, used by Hippocrates himself.

Flaxseed oil can help protect against cardiovascular disease, cancer, arthritis and many other degenerative diseases. And though it's primarily a source of omega-3, the oil has the virtue of containing some other fatty acids as well, notably some omega-6's and some heart healthy omega-9's, providing a nice fatty acid balance.

There's been a long-standing controversy among health professionals over the relative virtues of flaxseed oil and fish oil. Fish oil contains two omega-3 fatty acids — EPA and DHA — which are absolutely critical to human health. Many people believe that those are the true superstars of the omega-3 community. And there lies the heart of the controversy.

Flaxseeds and flaxseed oil are one of the best sources on the planet for the important omega-3 fatty acid called alpha-linoleic acid. Alpha-linoleic acid is considered an essential fatty acid because the body can't make it — it has to be obtained from the diet. The body can then take that ALA and theoretically convert it to EPA and DHA. The problem is that it doesn't do a very efficient job of conversion.

Best estimates are that the body will convert between 8-20 percent of ALA to EPA, and only .5 to 9 percent into DHA. But this ceases to be a problem at all if you simply take a tablespoon of flaxseed oil every day. And it's a non-issue if you're taking both flaxseed and fish oil.

Here's why: one tablespoon of quality flax oil is 14 grams (14,000 mg) of which 55 percent (7,700 mg) is compromised of omegs-3's. If you convert 7 percent of that to EPA you get 539 mg EPA and if you convert even 5 percent to DHA you'd get 385 mg of DHA. So even if you were a vegetarian and using flaxseed oil as your only source of omega-3's you'd be OK, though I'd recommend a couple of tablespoons a day instead of one.

And conversion to EPA and DHA isn't the only reason to take flaxseeds or flaxseed oils. A recent article in Nutrition Reviews (June 2008) pointed out that ALA may have "independent, therapeutic properties similar to those of other omega-3 fatty acids" and "may also be of particular importance for neural development."

But the benefits of flax are not limited to the omega-3 content. Both the oil and the seeds (especially the seeds) are a great source of something called lignans which have a whole host of health benefits of their own, for both men and women.

Lignans have a protective effect against cancer, especially those that are hormone-sensitive like breast, uterine and prostate cancer.

Yes, if you put a gun to my head and told me I could only take fish oil or flaxseed oil, I'd probably choose fish oil. But truth be told I take them both, every single day. And if you won't or can't eat fish or take fish oil, then simply up your intake of flaxseed oil.

A recent study of male firefighters found that only 2.4 to 3.6 grams of flaxseed oil alone raised the firefighters EPA levels. That's about 1/4 to 1/3 of a tablespoon. If you upped that dose a bit to just the one or two tablespoons a day I recommend, I'm willing to bet both EPA and DHA levels would rise.

It is my sincere hope that each of you take some step, however small, to improve your health and lifestyle over this next year.

God bless and have a perfect New Year.

Tuesday, October 2, 2012

Creating a JAR File

The basic format of the command for creating a JAR file is:
jar cf jar-file input-file(s)
The options and arguments used in this command are:
  • The c option indicates that you want to create a JAR file.
  • The f option indicates that you want the output to go to a file rather than to stdout.
  • jar-file is the name that you want the resulting JAR file to have. You can use any filename for a JAR file. By convention, JAR filenames are given a .jar extension, though this is not required.
  • The input-file(s) argument is a space-separated list of one or more files that you want to include in your JAR file. The input-file(s) argument can contain the wildcard * symbol. If any of the "input-files" are directories, the contents of those directories are added to the JAR archive recursively.
The c and f options can appear in either order, but there must not be any space between them.
This command will generate a compressed JAR file and place it in the current directory. The command will also generate a default manifest file for the JAR archive.

Note: The metadata in the JAR file, such as the entry names, comments, and contents of the manifest, must be encoded in UTF8.

You can add any of these additional options to the cf options of the basic command:
jar command options
OptionDescription
vProduces verbose output on stdout while the JAR file is being built. The verbose output tells you the name of each file as it's added to the JAR file.
0(zero)Indicates that you don't want the JAR file to be compressed.
MIndicates that the default manifest file should not be produced.
mUsed to include manifest information from an existing manifest file. The format for using this option is:
jar cmf existing-manifest jar-file input-file(s)
See Modifying a Manifest File for more information about this option.

Warning: The manifest must end with a new line or carriage return. The last line will not be parsed properly if it does not end with a new line or carriage return.

-CTo change directories during execution of the command. See below for an example.

Note: When you create a JAR file, the time of creation is stored in the JAR file. Therefore, even if the contents of the JAR file do not change, when you create a JAR file multiple times, the resulting files are not exactly identical. You should be aware of this when you are using JAR files in a build environment. It is recommended that you use versioning information in the manifest file, rather than creation time, to control versions of a JAR file. See the Setting Package Version Information section.


An Example

Let us look at an example. A simple TicTacToe applet. You can see the source code of this applet by downloading the JDK Demos and Samples bundle from Java SE Downloads. This demo contains class files, audio files, and images having this structure:
TicTacToe folder Hierarchy
TicTacToe folder Hierarchy
The audio and images subdirectories contain sound files and GIF images used by the applet.
You can obtain all these files from jar/examples directory when you download the entire Tutorial online. To package this demo into a single JAR file named TicTacToe.jar, you would run this command from inside the TicTacToe directory:
jar cvf TicTacToe.jar TicTacToe.class audio images
The audio and images arguments represent directories, so the Jar tool will recursively place them and their contents in the JAR file. The generated JAR file TicTacToe.jarwill be placed in the current directory. Because the command used the v option for verbose output, you would see something similar to this output when you run the command:
adding: TicTacToe.class (in=3825) (out=2222) (deflated 41%)
adding: audio/ (in=0) (out=0) (stored 0%)
adding: audio/beep.au (in=4032) (out=3572) (deflated 11%)
adding: audio/ding.au (in=2566) (out=2055) (deflated 19%)
adding: audio/return.au (in=6558) (out=4401) (deflated 32%)
adding: audio/yahoo1.au (in=7834) (out=6985) (deflated 10%)
adding: audio/yahoo2.au (in=7463) (out=4607) (deflated 38%)
adding: images/ (in=0) (out=0) (stored 0%)
adding: images/cross.gif (in=157) (out=160) (deflated -1%)
adding: images/not.gif (in=158) (out=161) (deflated -1%)
You can see from this output that the JAR file TicTacToe.jar is compressed. The Jar tool compresses files by default. You can turn off the compression feature by using the 0(zero) option, so that the command would look like:
jar cvf0 TicTacToe.jar TicTacToe.class audio images
You might want to avoid compression, for example, to increase the speed with which a JAR file could be loaded by a browser. Uncompressed JAR files can generally be loaded more quickly than compressed files because the need to decompress the files during loading is eliminated. However, there is a tradeoff in that download time over a network may be longer for larger, uncompressed files.
The Jar tool will accept arguments that use the wildcard * symbol. As long as there weren't any unwanted files in the TicTacToe directory, you could have used this alternative command to construct the JAR file:
jar cvf TicTacToe.jar *
Though the verbose output doesn't indicate it, the Jar tool automatically adds a manifest file to the JAR archive with path name META-INF/MANIFEST.MF. See the Working with Manifest Files: The Basics section for information about manifest files.
In the above example, the files in the archive retained their relative path names and directory structure. The Jar tool provides the -C option that you can use to create a JAR file in which the relative paths of the archived files are not preserved. It's modeled after TAR's -C option.
As an example, suppose you wanted to put audio files and gif images used by the TicTacToe demo into a JAR file, and that you wanted all the files to be on the top level, with no directory hierarchy. You could accomplish that by issuing this command from the parent directory of the images and audio directories:
jar cf ImageAudio.jar -C images . -C audio .
The -C images part of this command directs the Jar tool to go to the images directory, and the . following -C images directs the Jar tool to archive all the contents of that directory. The -C audio . part of the command then does the same with the audio directory. The resulting JAR file would have this table of contents:
META-INF/MANIFEST.MF
cross.gif
not.gif
beep.au
ding.au
return.au
yahoo1.au
yahoo2.au
By contrast, suppose that you used a command that did not employ the -C option:
jar cf ImageAudio.jar images audio
The resulting JAR file would have this table of contents:
META-INF/MANIFEST.MF
images/cross.gif
images/not.gif
audio/beep.au
audio/ding.au
audio/return.au
audio/yahoo1.au
audio/yahoo2.au

Saturday, September 29, 2012

Maven Repositories

Search Public Maven Artifacts:
http://search.maven.org/
http://download.java.net/maven/2
http://download.java.net/maven/1
http://repo.maven.apache.org/maven2


Spring Framework:
Releases:
<repository>  
  <id>com.springsource.repository.bundles.release</id>  
  <name>SpringSource Enterprise Bundle Repository - SpringSource Bundle Releases</name>  
  <url>http://repository.springsource.com/maven/bundles/release</url> 
</repository>
...
<dependency>  
  <groupId>org.springframework</groupId>  
  <artifactId>org.springframework.core</artifactId>  
  <version>3.0.2.RELEASE</version> 
</dependency>

Milestone:
<repositories>
 <repository>
  <id>springsource maven repo</id>
  <url>http://maven.springframework.org/milestone</url>
 </repository>
</repositories>
...
<dependencies>
  <dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-core</artifactId>
    <version>3.0.0.RC1</version>
  </dependency>
</dependencies>



Hibernate Framework:
<repositories>
    <repository>
        <id>jboss-public-repository-group</id>
        <name>JBoss Public Maven Repository Group</name>
        <url>https://repository.jboss.org/nexus/content/groups/public/</url>
        <layout>default</layout>
        <releases>
            <enabled>true</enabled>
            <updatePolicy>never</updatePolicy>
        </releases>
        <snapshots>
            <enabled>true</enabled>
            <updatePolicy>never</updatePolicy>
        </snapshots>
    </repository>
</repositories>
<pluginRepositories>
    <pluginRepository>
        <id>jboss-public-repository-group</id>
        <name>JBoss Public Maven Repository Group</name>
        <url>https://repository.jboss.org/nexus/content/groups/public/</url>
        <layout>default</layout>
        <releases>
            <enabled>true</enabled>
            <updatePolicy>never</updatePolicy>
        </releases>
        <snapshots>
            <enabled>true</enabled>
            <updatePolicy>never</updatePolicy>
        </snapshots>
    </pluginRepository>
</pluginRepositories>


Here is the dependency you need:
    <dependency>
        <groupId>org.hibernate</groupId>
        <artifactId>hibernate-core</artifactId>
        <version>3.5.4-Final</version>
    </dependency>

Following dependencies is required for slf4j error if you are using hibernate:
<dependency>
 <groupId>org.slf4j</groupId>
 <artifactId>slf4j-log4j12</artifactId>
 <version>1.5.6</version>
</dependency>

Tuesday, September 25, 2012

Avoiding the "javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed" error

Please note: this post focuses on the standard Java https implementation.

The SSLHandshakeException is thrown by java when the host you are trying to contact doesn't have a valid SSL certificate for that hostname. Most of the time this is very useful, since it means something on that host is wrong (the certificate has expired, the machine you're contacting is not who it is pretending to be etc...). However, in development mode you often don't want to pay for a "real" certificate, signed by a CA (certificate authority) like Verisign. You will then use a self-signed certificate, which gets rejected by java. It's for these cases that we're going to build a workaround. Please note that you should probably not use this code in a production environment. If you do, there's no reason to use https, since you're bypassing its functionality and you might just as well stick to http.

The first thing we need to do is create a custom TrustManager for SSL. SSL uses a protocol called X.509.  We will build a TrustManager that trusts all servers:
X509TrustManager tm = new X509TrustManager() { 
  @Override
  public X509Certificate[] getAcceptedIssuers() {
    return null;
  }  
  @Override
  public void checkServerTrusted(X509Certificate[] paramArrayOfX509Certificate, String paramString) throws CertificateException {   }  
  @Override
  public void checkClientTrusted(X509Certificate[] paramArrayOfX509Certificate, String paramString) throws CertificateException { }
};


As you can see, the checkXXXTrusted() methods throw Exceptions when something is wrong. We never throw an exception, effectively trusting all hosts.

The next thing we'll need to do is use this TrustManager on an SSLContext. An SSLContext is a factory class that is used to create socket factories, which in their turn create the actual ssl sockets used to communicate with the server. Here's how we do this:

SSLContext ctx = SSLContext.getInstance("TLS"); 
ctx.init(null, new TrustManager[] { tm }, null);
SSLContext.setDefault(ctx);


There now remains one more thing to be done: set a custom HostnameVerifier. A HostnameVerifier is a class that makes sure the host you are contacting doesn't use a spoofed URL. We will again build a HostnameVerifier that trusts all hosts:
HttpsURLConnection conn = (HttpsURLConnection) new URL("https://serverAddress").openConnection(); 
conn.setHostnameVerifier(new HostnameVerifier() {  
 @Override
 public boolean verify(String paramString, SSLSession paramSSLSession) {
  return true;
 }
});


Again, this HostnameVerifier will trust all hosts.
Putting all our code together, the final class will look like this:


public static void main(String[] args) throws NoSuchAlgorithmException, KeyManagementException, MalformedURLException, IOException {
 X509TrustManager tm = new X509TrustManager() {
  @Override
  public X509Certificate[] getAcceptedIssuers() {
   return null;
  }  
  @Override
  public void checkServerTrusted(X509Certificate[] paramArrayOfX509Certificate, String paramString) throws CertificateException {   }  
  @Override
  public void checkClientTrusted(X509Certificate[] paramArrayOfX509Certificate, String paramString) throws CertificateException { }
 };
 SSLContext ctx = SSLContext.getInstance("TLS");
 ctx.init(null, new TrustManager[] { tm }, null);
 SSLContext.setDefault(ctx);  
 HttpsURLConnection conn = (HttpsURLConnection) new URL("https://serverAddress").openConnection();
 conn.setHostnameVerifier(new HostnameVerifier() {  
  @Override
  public boolean verify(String paramString, SSLSession paramSSLSession) {
   return true;
  }
 });
 conn.connect();  
}


One final note: I prefer the way the Apache HttpClient library handles this. In the HttpClient library you can make a clean separation between the ssl verification logic and the code that does the actual work. This allows you to easily remove the code in the production environment or to use a switch between the development and production environment. This is much harder in the plain java version, since the code is more entangled. See this post for how to do this with the Apache HttpClient.
 

Avoiding the "javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated" with HttpClient

Please note: This article focuses on the Apache HttpClient library.

When developing a https application, your test server often doesn't have a (valid) SSL certificate. This will cause the following exception to be thrown when connecting your client to the test server: "javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated".
I will be discussing a way to fix this issue with the apache HttpClient, version 4.0.1

Bits and pieces
--------------------
You usually create your HttpClient like this:
client = new DefaultHttpClient();

We will need to tell the client to use a different TrustManager. A TrustManager is a class that checks if given credentials (or certificates) are valid. The scheme used by SSL is called X.509, and Java has a specific TrustManager for this scheme, called X509 TrustManager. First thing we will need to do is create such a TrustManager:
X509TrustManager tm = new X509TrustManager() {  
 public void checkClientTrusted(X509Certificate[] xcs, String string) throws CertificateException { }  
 public void checkServerTrusted(X509Certificate[] xcs, String string) throws CertificateException { }  
 public X509Certificate[] getAcceptedIssuers() {
 return null;
 }
};


As you can see, this code doesn't do much: if a certificate is invalid the TrustManager is supposed to throw a CertificateException in the checkXXX methods. Since we always want to accept all certificates, we never throw an exception.

Next we need to find a way to set this TrustManager in our HttpClient. The TrustManager is used by the SSL sockets. Sockets are created using a SocketFactory. For SSL sockets this is an SSLSocketFactory.

When creating a new SSLSocketFactory, you need to pass an SSLContext to the constructor. It is this SSLContext that will contain our newly created TrustManager.

First thing we need to do is get an SSLContext:SSLContext ctx = SSLContext.getInstance("TLS");

TLS is the successor to SSL, but they use the same SSLContext.
Then we initialize this context with our new TrustManager that we created above:ctx.init(null, new TrustManager[]{tm}, null);

We can then finally create our SSLSocketFactory:SSLSocketFactory ssf = new SSLSocketFactory(ctx);

Now we still need to register this SSLSocketFactory with our HttpClient. This is done in the SchemeRegistry of the ConnectionManager of the HttpClient:ClientConnectionManager ccm = base.getConnectionManager();
SchemeRegistry sr = ccm.getSchemeRegistry();
sr.register(new Scheme("https", ssf, 443));


We register a new Scheme, with the protocol https, our newly created SSLSocketFactory which contains our TrustManager and we tell the HttpClient that the default port for https is port 443.

Putting it all together:
===================
The following class takes a HttpClient and returns a new HttpClient that accepts any SSL certificate:
/* This code is public domain: you are free to use, link and/or modify it in any way you want, for all purposes including commercial applications.  */
public class WebClientDevWrapper { 
 public static HttpClient wrapClient(HttpClient base) {
 try { 

  SSLContext ctx = SSLContext.getInstance("TLS");
  X509TrustManager tm = new X509TrustManager() {  
   public void checkClientTrusted(X509Certificate[] xcs, String string) throws CertificateException { }  
   public void checkServerTrusted(X509Certificate[] xcs, String string) throws CertificateException { }  
   public X509Certificate[] getAcceptedIssuers() {
    return null;
   }
  }; 


  ctx.init(null, new TrustManager[]{tm}, null);
  SSLSocketFactory ssf = new SSLSocketFactory(ctx);
  ssf.setHostnameVerifier(SSLSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);
  ClientConnectionManager ccm = base.getConnectionManager();
  SchemeRegistry sr = ccm.getSchemeRegistry();
  sr.register(new Scheme("https", ssf, 443));
  return new DefaultHttpClient(ccm, base.getParams());
 } catch (Exception ex) {
  ex.printStackTrace();
  return null;
 }
      }
}


You can then do something like this in the code that creates the HttpClient:
this.client = new DefaultHttpClient();
if(dev) {
 this.client = WebClientDevWrapper.wrapClient(client);
}

- - - - - - - - - - - - - - - - - - - - - - - - - - - -  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Update:
In some exceptional cases, the method described above doesn't work. This is due to the Apache AllowAllHostnameVerifier still being to strict. In this case, you will need your own X509HostnameVerifier. Create it as follows:
X509HostnameVerifier verifier = new X509HostnameVerifier() {                  
 @Override               
 public void verify(String string, SSLSocket ssls) throws IOException {}                  
 @Override               
 public void verify(String string, X509Certificate xc) throws SSLException {}
 @Override               
 public void verify(String string, String[] strings, String[] strings1) throws SSLException {}
 @Override               
 public boolean verify(String string, SSLSession ssls) {
  return true;                
 }            
};


Then set it on your socket factory:
ssf.setHostnameVerifier(verifier);

If we put everything together, the new code looks like this:
/* This code is public domain: you are free to use, link and/or modify it in any way you want, for all purposes including commercial applications.  */

public class WebClientDevWrapper {      
 public static HttpClient wrapClient(HttpClient base) {        
  try {             

   SSLContext ctx = SSLContext.getInstance("TLS");
   X509TrustManager tm = new X509TrustManager() {                  
    public void checkClientTrusted(X509Certificate[] xcs, String string) throws CertificateException {}
    public void checkServerTrusted(X509Certificate[] xcs, String string) throws CertificateException {}
    public X509Certificate[] getAcceptedIssuers() {                    
     return null;                
    }            
   };

            
   X509HostnameVerifier verifier = new X509HostnameVerifier() {                   

   @Override               
   public void verify(String string, SSLSocket ssls) throws IOException {}

   @Override               
   public void verify(String string, X509Certificate xc) throws SSLException {}

   @Override               
   public void verify(String string, String[] strings, String[] strings1) throws SSLException {}

   @Override               
   public boolean verify(String string, SSLSession ssls) {
    return true; 
   }            
  };             

  ctx.init(null, new TrustManager[]{tm}, null);            
  SSLSocketFactory ssf = new SSLSocketFactory(ctx);            
  ssf.setHostnameVerifier(verifier);            
  ClientConnectionManager ccm = base.getConnectionManager();    
  SchemeRegistry sr = ccm.getSchemeRegistry();            
  sr.register(new Scheme("https", ssf, 443));            
  return new DefaultHttpClient(ccm, base.getParams());         

 } catch (Exception ex) {            
  ex.printStackTrace();            
  return null;        
 }    
    }
}

You can then do something like this in the code that creates the HttpClient:
this.client = new DefaultHttpClient();
if(dev) {
 this.client = WebClientDevWrapper.wrapClient(client);
}

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Working Example implemented in one of my project:

package com.company.test;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import org.apache.http.HttpResponse;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.conn.scheme.Scheme;
import org.apache.http.conn.scheme.SchemeRegistry;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.conn.ssl.SSLSocketFactory;
import org.apache.http.conn.ssl.TrustStrategy;

import java.security.KeyManagementException;
import java.security.KeyStoreException;
import java.security.NoSuchAlgorithmException;
import java.security.UnrecoverableKeyException;
import java.security.cert.CertificateException;
import java.security.cert.X509Certificate;


public class HttpsTestClient {
 public static void main(String[] args) throws KeyManagementException, UnrecoverableKeyException, NoSuchAlgorithmException, KeyStoreException {
  SchemeRegistry registry = new SchemeRegistry();
  SSLSocketFactory socketFactory = new SSLSocketFactory(new TrustStrategy() {
   public boolean isTrusted(X509Certificate[] arg0, String arg1) throws CertificateException {
    return true;
   }
  }, org.apache.http.conn.ssl.SSLSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);
 
  registry.register(new Scheme("https", 443, socketFactory));
  ThreadSafeClientConnManager mgr = new ThreadSafeClientConnManager(registry);
 
  DefaultHttpClient client = new DefaultHttpClient(mgr, new DefaultHttpClient().getParams());
       
     HttpPost post = new HttpPost("
https://host:7032/Xyz/service");
     try {
      String send = "< ?xml version=\"1.0\" encoding=\"UTF-8\"?> <msg> <head><Client>ComXCalFL</Client> <Region>QAYA</Region> <Source>E26XCALIBUR</Source><Destination>CCS01</Destination> <Origin>L0OG1-X-CAL ORDER DETAIL UPDATED</Origin> <Version>2.25</Version> <RoundTrip>MDU6MDAwMDAwMDM6MjAxMjA4MzE6MTE0OTE0MDA6ODIxMDAwMDA6TDBPRzowMg==</RoundTrip> <UserId>LRM 0B72</UserId> <ServerId>sbappre02-z04</ServerId> <TriggerDate>2012-08-31</TriggerDate><TriggerTime>11:49:14</TriggerTime> </head><body><OrderDetailUpdated> XML_PAYLOAD</OrderDetailUpdated></body></msg>";

     
      StringEntity strEnt = new StringEntity(send);
      post.setEntity(strEnt);
       HttpResponse response = client.execute( post);
       BufferedReader rd = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
       String line = "";
       while ((line = rd.readLine()) != null) {
         System.out.println(line);
       }

     } catch (IOException e) {
       e.printStackTrace();
     }
   }

}



Thursday, July 19, 2012

Apache JMeter

Apache J Meter

# - Remember to set JAVA_HOME to JDK 5+
Example:
Server 1:
JAVA_HOME=/app/weblogic/jdk1.6.0_33/bin

WEBLOGIC_HOME=/app/home/mradmin/Oracle/Middleware
APACHE_HOME=/app/cemp/apache-jmeter-2.7
PATH=/app/weblogic/jdk1.6.0_33/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/app/home/mradmin/bin:/app/cemp/apache-jmeter-2.7/bin/ApacheJMeter.jar:
export PATH
Server 2: [Same on Server 2]

# - [cemp@cempda01 bin]$ jmeter -n -r -t Usage\ Cache\ Users.jmx -LDEBUG -l results.jtl
Example #:

[cemp@cempda01 bin]$ jmeter -n -r -t UsageCacheUsers.jmx -LDEBUG -l results.jtl
Created the tree successfully using UsageCacheUsers.jmx

Configuring remote engine for cmpcol-dt-4d.ula.com.net
Using remote object: UnicastRef [liveRef: [endpoint:[10.255.156.129:49889](remote),objID:[71770134:138a12bc3d9:-7fff, 5263036063349195177]]]
Configuring remote engine for cempda01.cab.com.com
Using remote object: UnicastRef [liveRef: [endpoint:[147.191.113.124:34820](remote),objID:[584b8767:138a1409048:-7fff, -1613329581385448862]]]
Starting remote engines
Starting the test @ Fri Jul 20 05:29:06 UTC 2012 (1342762146778)
Remote engines have been started
Waiting for possible shutdown message on port 4445
Generate Summary Results + 916 in 31.3s = 29.3/s Avg: 34 Min: 21 Max: 176 Err: 0 (0.00%)
Generate Summary Results + 2631 in 89.0s = 29.6/s Avg: 35 Min: 20 Max: 134 Err: 0 (0.00%)
Generate Summary Results = 3547 in 120.3s = 29.5/s Avg: 35 Min: 20 Max: 176 Err: 0 (0.00%)
Tidying up remote @ Fri Jul 20 05:31:29 UTC 2012 (1342762289011)
... end of run
[cemp@cempda01 bin]$
# - Important Commands:
1. [cemp@cempda01 bin]$ netstat -tulpn // Check port - process id.
2. Microsoft Windows:  start rmiregistry 30031 // remember to JAVA_HOME before.
3. Solaris OS or Linux:  rmiregistry 30031 & // remember to set JAVA_HOME before.
By default, the registry runs on port 1099. To start the registry on a different port, specify the port number on the command line. Do not forget to unset your CLASSPATH environment variable.
To check that the RMI registry has started correctly, you can check that it is listening on its standard port (1099 unless you've specified otherwise) with a tool such as TCP View.






 
 
 

Wednesday, July 18, 2012

Mozilla Unresponsive Script

Any Solution ?

Warning: Unresponsible Script

A script on this page may be busy , or it may have stopped responding. You can stop the script now, or you can continue to see if the script will complete.
Script: http://s.ytimg.com/yt/jsbin/www-core-vf19FZ5Nr.js:219

Sunday, June 17, 2012

CVS Quick Reference

CVS Options
Use the following syntax to issue CVS Commands:
$ cvs [cvs-option] command [cmd-options] [args]

Replace cvs-options with one  or more of the following:
1. -a : Authenticate the data stream
2. --allow-root=directory : Set the repository root directory (used in server modes)
3. -d repository-path : Specify the repository path for the current command
4. -e editor : Use the specified editor in commit or import commands.
5. -f : Do not use the ~/.cvsrc file
6. -H or --help : Display help for CVS
7. --help-commands : Display a list of CVS Commands.
8. --help-options : Display a list of CVS Options.
9. --help-synonyms : Display a list of synonyms for CVS commands.
10. --H command or --help command : Display the options and help for the specified command.
11. -l : Do not log the current command to the history file.
12. -n : do not write to the repository or sandbox.
13. -q : Run in quiet mode.
14. -Q : Run in very quiet mode.
15. -r : Set files checked out to the sandbox read-only.
16. -s : variable=value : Set a user variable for use with one of the scripting files in CVSROOT
17. -t: Display messages that trance the execution of the command.
18. -v or --version : Display CVS version and copyright information.
19. -w : Set files checked out to the sandbox readable and writable.
20. -x : Encrypt all data that travels across the network.
21. -z # : Compress all network traffic using the specified gzip compression level.

Repository Access:
Repository path information is as follows:
[:method] [[[user][:password]@]hostname[:[port]]]/path 

Access method can be any of the following:
1. ext: Connection through external rsh or rsh-like program.
2. fork: Client / server mode on local computer.
3. gserver : Connection through the GSS-API and Kerberos 5
4. local : Local computer, no client / server mode.
5. pserver : Connection through password server.
6. kserver: Connection through kerberos 4.
7. server: Connection through internal rsh server key environment variables relating to repository access are
     CVS_RSH: Specifies rsh-like executable.
     CVSROOT: Specifies repository path.

Common CVS Tasks:
The following lists show the syntax to use in performing common CVS tasks. Remember that for any CVS command you can specify the repository path using the -d option. For example:
$ cvs -d repository_path checkout project
For brevity, the -d option is not shown in any of the example commands. However, it can be added to any of the commands.

Sandbox Commands:
Create a new sandbox: $ cvs checkout project
Remove a sandbox: $ cvs release directory
Check out files as of a specific tag name or revision: cvs checkout -r [tagname | revision] project
Check out files as of a specific date or time:  $ cvs checkout -D [date|time] project

Synchronization Commands:
Upload changes to the repository: $ cvs commit [filenames]
Force upload of unchanged files: $ cvs commit -f [filenames]
Download changes from the repository: $ cvs update [filenames]
Download changes and download new directories: $ cvs update -d [filenames]
Download changes. but not empty directories : $ cvs update -P [filenames]
Download changes, clear sticky revision, date, keyword mode or branch: $ cvs update -A [filenames]
Convert an existing sandbox to a specific tag name or revision: $ cvs update -r [tagname | revision] project
Convert an existing sandbox to a specific date or time: $ cvs update -D [date|time] project

File Commands:
Add new files or directories to the repository: $ cvs add [filenames]
Add new binary files to the repository: $ cvs add -kb [filenames]
Remove files or directories from the repository: $ cvs remove [filenames]

Branch Commands:
Create a branch from within a sandbox: $ cvs tag -b branchname [filenames]
Create a branch from anywhere using a tag name or revision: $ cvs rtag -r [tagname|revision] -b branchname project
Create a branch from anywhere using a date or time: $ cvs rtag -D [date|time] branchname project
Checkout a branch sandbox: $ cvs checkout -r branchname project
Convert an existing sandbox to a branch sandbox: $ cvs update -r branchname [filenames]

Tag Commands:
Tag files from within a sandbox: $ cvs tag tagname [filenames]
Tag files from anywhere using a tag name or revision: $ cvs rtag -r [existing_tagname | revision] new_tagname project
Tag files from anywhere using a date or time: $ cvs rtag -D [date|time] tagname project
Move a tag from within a sandbox: $ cvs tag -F tagname [filenames]
Move a tag from anywhere using a tag name or revision: $ cvs rtag -r [tagname_at_location|revision] -F tagname_to_more project
Move a tag form anywhere using a date or time: $ cvs rtag -D [date|time] -F tagname project
Delete a tag from within a sandbox: $ cvs tag -d tagname [filenames]
Delete a tag from anywhere: $cvs rtag -d tagname project

The -r [tagname|revision] option accepts a tag name or revision, but cannot have both tag name and revision. The -D [date|time] option accepts a date, a time, or both date and time.

CVSROOT: :ssh:adixit3034c@cmputl-po-3p:/opt/cm/cmsys/cvs/root
Server: cmputl-po-3p
Repository Folder: /opt/cm/cmsys/cvs/root

Username: adixit3034c

[Working]$ cvs -d adixit3034c@cmputl-po-3p:/opt/cm/cmsys/cvs/root checkout -P -r CFX_BR_EEPBill_DDPMediation_2_0_0_5 CFX_BR_EEPBill_DDPMediation

$ cd /data/Release_elements/current/US242513
$ tar -czvf CFX_BR_EEPBill_DDPMediation_2_0_0_5.tar.gz CFX_BR_EEPBill_DDPMediation_2_0_0_5

$ chmod 755 CFX_BR_EEPBill_DDPMediation_2_0_0_5 -R
$ cd /data/Release_elements/current/US242513/CFX_BR_EEPBill_DDPMediation_2_0_0_5/_EEPBill_DDPMediation/app

$ mvn clean install -Dmaven.test.skip=true

Location: http://cmputl-po-4p/cm_workareas/current/