Windows HPC Community - Developer Blog

7/2/2008

One of the big challenges as we get close to shipping Windows HPC Server is closing the door on product changes. Now that we have a lot of internal and external customers using the product, everybody generates lots of great ideas on how to improve it. We call these DCRs (Design Change Request) and we spend a lot of time arguing over whether we should accept them (creating additional design, programming, test, documentation, and localization work) or reject them (shipping a product which might suffer from anything between small annoyances and large deployment blockers or support issues) As we get closer and closer to RTM, the bar for accepting DCRs gets higher. This is the story of one DCR we rejected.

A developer on my team filed a DCR asking for a Powershell cmdlet that returned a list of all the HPC clusters in the domain. This would be handy for a lot of different reasons. We debated this a bit and eventually rejected it because it’s not really specific to the HPC product and because you can already do this pretty easily in Powershell. Powershell doesn’t have built-in support for ActiveDirectory, but you can call out to the .NET Framework and use the DirectorySearcher class.

PS C:\> $Searcher = New-Object DirectoryServices.DirectorySearcher

PS C:\> $Searcher.Filter = "(&(serviceClassName=MicrosoftComputeCluster)(keywords=Version2))"

PS C:\> $Searcher.FindAll()

Path Properties

---- ----------

LDAP://CN=MicrosoftComputeCluster,CN... {keywords, servicednsname, servicedn...

What you get back from $Searcher.FindAll() is a list of all the computer objects in the domain where the HPC job scheduler is running. Really all you care about here is the servicednsname property, so we can use ForEach-Object (abbreviated as foreach or %) to filter out the other stuff.

PS C:\> $schedlist = $Searcher.FindAll() | foreach {$_.Properties.servicednsname}

PS C:\> $schedlist

kyril.ntdev.corp.microsoft.com

37-4611A2803A.ntdev.corp.microsoft.com

JVERT.ntdev.corp.microsoft.com

GRET57001.ntdev.corp.microsoft.com

R25-1183D1023.ntdev.corp.microsoft.com

gret10.ntdev.corp.microsoft.com

liwei.ntdev.corp.microsoft.com

Now we have a list of all the HPC schedulers in the domain, we can stuff that into the Get-HpcClusterOverview cmdlet and collect their individual states:

PS C:\> $overview = $schedlist | foreach {get-hpcclusteroverview -scheduler $_}

If there are a lot of clusters in your domain that are unreachable, this might take a while because Get-HpcClusterOverview will wait for each connection to time-out. Once it does complete, $overview will have a list containing a snapshot of the statistics for each active cluster. You can pipe that through format-table or do whatever you like with it.

PS C:\> $overview | format-table

Cluster Version TotalNo ReadyNo Offline Drainin Unreach TotalCo BusyCor IdleCor

Name deCount deCount NodeCou gNodeCo ableNod reCount eCount eCount

nt unt eCount

------- ------- ------- ------- ------- ------- ------- ------- ------- -------

kyrilfl 2.0.... 1 1 0 0 0 2 0 2

37-4... 2.0.... 4 4 0 0 0 32 0 32

JVERT2 2.0.... 1 1 0 0 0 2 0 2

GRET... 2.0.... 1 0 1 0 0 8 0 0

R25-... 2.0.... 3 3 0 0 0 20 0 20

gret10. 2.0.... 1 0 1 0 0 8 0 0

liwei.. 2.0.... 1 1 0 0 0 2 0 2

I think this is a great example of how to use Powershell to easily glue together two different systems (AD and HPC) in an interactive way. I hope this will give you a little taste of some of the neat things you can do with the Powershell support in HPC as well as a little taste of what the end-game of shipping a product at Microsoft involves.

John Vert

Development Manager

High Performance Computing

Posted at 6:26 PM by John Vert | Category: Development | Permalink | Email this Post | Comments (0)

6/5/2008

Converting an iterative sequential application to data parallel using CMD scripts

I have an application that is based on a sequential iterative call from a command script. It is a Monte Carlo simulation application is essentially easy to put in a data parallel format and deployed to run on CCS cluster easily.

Desciption of the sequential iterative version:

A main.exe program generates the n inputs for variation runs. The program has three inputs: Directory for results output, Project name, Number of variations
For example:
main Results2008 test_ccs 100

A DOS script (run_sub_DOS.bat) called by main execute the ‘distributed’ program n times iteratively. it calls the application "simulation" iteratively.

this is how the script looks:

@echo off

FOR /L %%i IN (1,1,%3) DO simulation %1 %2.%%i%

To modify this into a data parallel version I first changed this script to the following (call it run_sub_CCS.bat). This script, as mentioned above in the sequential case, is called by "main.exe".

set workdir=\\RRS-HN\lhuang\RRS-Example\Test1
FOR /F "usebackq tokens=4" %%j in (`job new /numprocessors:4-4 /jobname:RRS_TESTjob`) do set JobID=%%j
FOR /L %%i IN (1,1,%3) DO job add %JobID% /workdir:%workdir% /stdout:frf.out.%%i simulation %1 %2.%%i%

Then I had the following two wrap scripts to call this. That is I calle the scripts - CCS_Begin (which calls main) and CCS_Finalize in that order. These does some extra work of copying data to the CCS nodes. I decided to leave them as it is to point out the real environment.

@Rem CCS_Begin: script to run main

@call clusrun /readynodes rmdir \\localhost\C$\ranjan\RRS /Q /S

@call clusrun /readynodes mkdir \\localhost\C$\ranjan\RRS\Sub

@call clusrun /readynodes xcopy \\RRS-HN\ranjan\RRS\Sub\\localhost\C$\ranjan\RRS /Y /S

main Results2008 test_css 4

@FOR /F "usebackq tokens=1" %%k in (`job list /status:NotSubmitted`) do set JobID=%%k

job submit /ID:%JobID%

@Rem CCS_Finalize

FOR /F "usebackq tokens=1" %%k in (`job list /status:finished`) do set JobID=%%k
if %JobID%==%1 call clusrun /readynodes rmdir \\localhost\C$\ranjan\RRS /Q /S

(adopted from real application scenario).

Posted at 5:09 PM by ranjan | Category: Development | Permalink | Email this Post | Comments (0)

3/7/2008

clusrun.vbs fix to handle name="value" parameters

I want to install an application on a CCS cluster nodes using MSIEXEC. I use clusrun while have the package available on a share. i want the installation target to be at C:\Program Files\product name location on all nodes.

clusrun hangs when I give TARGETDIRECTORY="Program Files\product name" as a parameter for MSIEXEC. A simple workaround I used:

change the clusrun.vbs as follows -

(look at <CCP ROOT>\bin\ for clusrun.vbs).

you need to look at the part that processes supplied arguments in the commnd line (line 101 through 144). It should be as follows.

for i=0 to objArgs.Count - 1
if Left(objArgs(i),1) <> "/" then

          if schedname="" then
           set wshShell = Wscript.CreateObject ("WScript.Shell")
             set WshEnv = WshShell.Environment("Process")
             schedname = WshEnv("CCP_SCHEDULER")
             if schedname="" then
                 schedname = "localhost"
             end if
         end if

         '
         ' build the command line out of everything left
         '
         for j = i to objArgs.Count - 1
            'if the argument contains a space then add quotes to maintain the structure

          if InStr(objArgs(j), " ") = 0 then
             cmdline = cmdline + objArgs(j) + " "
           else
                cmdline = cmdline + Replace(objArgs(j),"=","=""") + """ "
           end if

         next
         task.CommandLine = cmdline
         exit for

end if

I could successfully use MSIEXEC with name value parameter and quotes as well as spaces.

Note: One way to overcome situations of having space in paramter is to place the command within a batch file and execute the batch file using clusrun or job submit.

Posted at 6:31 PM by ranjan | Category: Development | Permalink | Email this Post | Comments (0)

2/28/2008

Windows Communication Foundation (WCF) + HPC

You'll want to review a recent MSDN forum entry regarding this feature now available with Windows HPC Server 2008 (currently in beta). Read more here.

Posted at 7:47 PM by Phil | Category: Development | Permalink | Email this Post | Comments (0)

2/6/2008

getting JOBID in a perl script

The following article was the closest one I found when looking for scripting for CCS. I was told `new job ..` will return JOBID but not sure in what format!!- it is actually a string of 4 terms. Was not quite apparent from what was there on the example script which failed to run at the %CCS_JOBID% part (see URL below).

I came up with a hack to fix that - I think we can use scaler instead of the implicit way of coercing the string to number. So first I substr the number as string - then coerce it to number and use it! The hack I used (based on the script that uses %CCS_JOBID% at Look at the script posted on http://technet2.microsoft.com/windowsserver/en/library/30f7fbc9-af05-4bd4-97d5-aa9f97a3f6961033.mspx?mfr=true

# create new job
$id = `job new /numprocessors:4`;
# print $id;
$id = substr($id, 16,4) + 0;
# print $id;
# add parametric tasks
for ($i = 0; $i < 5 ; $i++) {
`job add $id /stdin:\\fileshare\infile$i.txt /stdout:outfile$i.txt myapp.exe`;
}

`job submit /ID:$id`;

Posted at 2:54 PM by ranjan | Category: Development | Permalink | Email this Post | Comments (0)

1/2/2008

Channel9 Video Presents New Job Scheduler OGF Web Service Interface

View the screen-cast here: http://channel9.msdn.com/ShowPost.aspx?PostID=369487

Posted at 1:30 PM by Phil | Category: Development | Permalink | Email this Post | Comments (0)

12/17/2007

Channel9 Video Presents the New Job Scheduler Features

Watch the screen-cast at http://channel9.msdn.com/ShowPost.aspx?PostID=366694.

Posted at 11:07 PM by Phil | Category: Development | Permalink | Email this Post | Comments (0)

Channel9 Video about the New Network Direct RDMA Interface

Check it out here:

http://channel9.msdn.com/ShowPost.aspx?PostID=366683

Posted at 8:39 PM by Phil | Category: Development | Permalink | Email this Post | Comments (0)

12/6/2007

HPC Application Deployment Scenarios and Redistributable Dependencies

Windows HPC Server 2008 Beta1 introduces 3 new programmatic interfaces to the Compute Cluster Scheduler. This effectively changes client application deployment scenarios along with client installation dependencies.

Please refer to my blog at http://blogs.msdn.com/philpenn/ for a more detailed commentary.

Posted at 7:43 AM by Phil | Category: Development | Permalink | Email this Post | Comments (0)

11/29/2007

Building native Windows binaries from UNIX code

Are you looking for a convenient and familiar way to port your UNIX code to Windows for use on the CCS platform? There are a few ways to do this. I have just uploaded a document that describes one of those methods using a shell script in conjunction with SUA.

This method hinges on the fact that it is completely permissible to create native Windows binary objects with MSVC from within the SUA environment.

This document provides some background about the evolution of the script, describes the use of the script and provides pointers to all the necessary components and setup required to use this method. It is really very straightforward.

Get the document here: https://windowshpc.net/Resources/Documents/c89.doc

Posted at 1:35 PM by Denniscr | Permalink | Email this Post | Comments (0)