Windows Vista Tips

Windows Vista Tips > Newsgroups > Windows Server > Scripting > using START to try and parallelize copy operations, getting hung u

Reply
Thread Tools Display Modes

using START to try and parallelize copy operations, getting hung u

 
 
Mark
Guest
Posts: n/a

 
      01-27-2010
Hi...

I have a batch file I'm launching from a web page using psexec.exe. The
last stage of the batch file is to copy a file to several locations. The
problem is that 2 of the 3 destinations are going through a network pipe the
size of a swizzle stick.

I was hoping to cut down on the pain by parallelizing the copies with START
(i.e. move the actual XCOPY command into its own little bat and have the
parent bat start them all at once).

I figured that the bottleneck is the network pipe, not the disk access to
the local file being copied so I might save some time by getting all the
copies going at once.

Problem is that when I ran this up the flagpole, the parent bat just hung at
the start command. Didn't copy anything. Is START something you shouldn't
use when you're already in the background? Or running under psexec?

Thanks
Mark

 
Reply With Quote
 
 
 
 
Pegasus [MVP]
Guest
Posts: n/a

 
      01-27-2010


"Mark" <> said this in news item
news:82165B35-9D63-4D84-841C-...
> Hi...
>
> I have a batch file I'm launching from a web page using psexec.exe. The
> last stage of the batch file is to copy a file to several locations. The
> problem is that 2 of the 3 destinations are going through a network pipe
> the
> size of a swizzle stick.
>
> I was hoping to cut down on the pain by parallelizing the copies with
> START
> (i.e. move the actual XCOPY command into its own little bat and have the
> parent bat start them all at once).
>
> I figured that the bottleneck is the network pipe, not the disk access to
> the local file being copied so I might save some time by getting all the
> copies going at once.
>
> Problem is that when I ran this up the flagpole, the parent bat just hung
> at
> the start command. Didn't copy anything. Is START something you
> shouldn't
> use when you're already in the background? Or running under psexec?
>
> Thanks
> Mark


This is probably easy to work out but my crystal ball is a little cloudy
right now. I've booked it for a cut & polish service early next week. Care
to post your batch file in the meantime?

 
Reply With Quote
 
Mark
Guest
Posts: n/a

 
      01-29-2010
The whole system is too large to post here, but the part I'm asking about is
a bat file spawned from a javascript .wsf file:
ECHO Copy call for %2
time /t
@IF NOT DEFINED StageServer GOTO END
start /b /high Copy.Machine1.bat %1 %2
start /b /high Copy.Machine2.bat %1 %2
start /b /high /wait cmd /c Copy.Machine3.bat %1 %2
time /t
:END

And each of the Copy bat files looks like this:
@ECHO Starting Machine001 copy
time/t
@NET USE \\Machine1\ /user:Machine1\%username%
XCOPY /Y "%1" "\\Machine1\%2"
@ECHO Done with Machine1 copy
time/t

Using Procexp.exe, I can see that the bat file is hung on the first start
command and never finishes. Nor is any file copied to Machine1.

Thanks
mark

"Pegasus [MVP]" wrote:

>
>
> "Mark" <> said this in news item
> news:82165B35-9D63-4D84-841C-...
> > Hi...
> >
> > I have a batch file I'm launching from a web page using psexec.exe. The
> > last stage of the batch file is to copy a file to several locations. The
> > problem is that 2 of the 3 destinations are going through a network pipe
> > the
> > size of a swizzle stick.
> >
> > I was hoping to cut down on the pain by parallelizing the copies with
> > START
> > (i.e. move the actual XCOPY command into its own little bat and have the
> > parent bat start them all at once).
> >
> > I figured that the bottleneck is the network pipe, not the disk access to
> > the local file being copied so I might save some time by getting all the
> > copies going at once.
> >
> > Problem is that when I ran this up the flagpole, the parent bat just hung
> > at
> > the start command. Didn't copy anything. Is START something you
> > shouldn't
> > use when you're already in the background? Or running under psexec?
> >
> > Thanks
> > Mark

>
> This is probably easy to work out but my crystal ball is a little cloudy
> right now. I've booked it for a cut & polish service early next week. Care
> to post your batch file in the meantime?
>

 
Reply With Quote
 
Pegasus [MVP]
Guest
Posts: n/a

 
      01-30-2010
To find out where your batch file gets stuck, use the syntax below, then
examine the log file c:\test.txt. It probably tells you everything you need
to know.

@echo off
ECHO Copy call for %2 on %date% at %time% (User=%UserName%) >> c:\Test.txt
IF NOT DEFINED StageServer GOTO :eof
start /b /high c:\Tools\Copy.Machine1.bat %1 %2
start /b /high c:\Tools\Copy.Machine2.bat %1 %2
start /b /high /wait cmd /c c:\tools\Copy.Machine3.bat %1 %2
ECHO Copy call for %2 on %date% at %time% >> c:\Test.txt

@echo off
ECHO Starting Machine001 copy on %date% at %time% >> c:\Test.txt
NET USE \\Machine1\ /user:Machine1\%username% 1>> c:\Test.txt 2>>&1
XCOPY /Y /c "%1" "\\Machine1\%2" 1>> c:\Test.txt 2>>&1
ECHO Done with Machine1 copy on %date% at %time% >> c:\Test.txt


"Mark" <> said this in news item
news:2570F589-CC65-42F9-921C-...
> The whole system is too large to post here, but the part I'm asking about
> is
> a bat file spawned from a javascript .wsf file:
> ECHO Copy call for %2
> time /t
> @IF NOT DEFINED StageServer GOTO END
> start /b /high Copy.Machine1.bat %1 %2
> start /b /high Copy.Machine2.bat %1 %2
> start /b /high /wait cmd /c Copy.Machine3.bat %1 %2
> time /t
> :END
>
> And each of the Copy bat files looks like this:
> @ECHO Starting Machine001 copy
> time/t
> @NET USE \\Machine1\ /user:Machine1\%username%
> XCOPY /Y "%1" "\\Machine1\%2"
> @ECHO Done with Machine1 copy
> time/t
>
> Using Procexp.exe, I can see that the bat file is hung on the first start
> command and never finishes. Nor is any file copied to Machine1.
>
> Thanks
> mark
>
> "Pegasus [MVP]" wrote:
>
>>
>>
>> "Mark" <> said this in news item
>> news:82165B35-9D63-4D84-841C-...
>> > Hi...
>> >
>> > I have a batch file I'm launching from a web page using psexec.exe.
>> > The
>> > last stage of the batch file is to copy a file to several locations.
>> > The
>> > problem is that 2 of the 3 destinations are going through a network
>> > pipe
>> > the
>> > size of a swizzle stick.
>> >
>> > I was hoping to cut down on the pain by parallelizing the copies with
>> > START
>> > (i.e. move the actual XCOPY command into its own little bat and have
>> > the
>> > parent bat start them all at once).
>> >
>> > I figured that the bottleneck is the network pipe, not the disk access
>> > to
>> > the local file being copied so I might save some time by getting all
>> > the
>> > copies going at once.
>> >
>> > Problem is that when I ran this up the flagpole, the parent bat just
>> > hung
>> > at
>> > the start command. Didn't copy anything. Is START something you
>> > shouldn't
>> > use when you're already in the background? Or running under psexec?
>> >
>> > Thanks
>> > Mark

>>
>> This is probably easy to work out but my crystal ball is a little cloudy
>> right now. I've booked it for a cut & polish service early next week.
>> Care
>> to post your batch file in the meantime?
>>

 
Reply With Quote
 
Mark
Guest
Posts: n/a

 
      02-01-2010
The output was already being captured farther up. It gets as far as the
start command, where it hangs. Nothing in the bat to be executed comes out,
nothing after the first start command.

"Pegasus [MVP]" wrote:

> To find out where your batch file gets stuck, use the syntax below, then
> examine the log file c:\test.txt. It probably tells you everything you need
> to know.
>
> @echo off
> ECHO Copy call for %2 on %date% at %time% (User=%UserName%) >> c:\Test.txt
> IF NOT DEFINED StageServer GOTO :eof
> start /b /high c:\Tools\Copy.Machine1.bat %1 %2
> start /b /high c:\Tools\Copy.Machine2.bat %1 %2
> start /b /high /wait cmd /c c:\tools\Copy.Machine3.bat %1 %2
> ECHO Copy call for %2 on %date% at %time% >> c:\Test.txt
>
> @echo off
> ECHO Starting Machine001 copy on %date% at %time% >> c:\Test.txt
> NET USE \\Machine1\ /user:Machine1\%username% 1>> c:\Test.txt 2>>&1
> XCOPY /Y /c "%1" "\\Machine1\%2" 1>> c:\Test.txt 2>>&1
> ECHO Done with Machine1 copy on %date% at %time% >> c:\Test.txt
>
>
> "Mark" <> said this in news item
> news:2570F589-CC65-42F9-921C-...
> > The whole system is too large to post here, but the part I'm asking about
> > is
> > a bat file spawned from a javascript .wsf file:
> > ECHO Copy call for %2
> > time /t
> > @IF NOT DEFINED StageServer GOTO END
> > start /b /high Copy.Machine1.bat %1 %2
> > start /b /high Copy.Machine2.bat %1 %2
> > start /b /high /wait cmd /c Copy.Machine3.bat %1 %2
> > time /t
> > :END
> >
> > And each of the Copy bat files looks like this:
> > @ECHO Starting Machine001 copy
> > time/t
> > @NET USE \\Machine1\ /user:Machine1\%username%
> > XCOPY /Y "%1" "\\Machine1\%2"
> > @ECHO Done with Machine1 copy
> > time/t
> >
> > Using Procexp.exe, I can see that the bat file is hung on the first start
> > command and never finishes. Nor is any file copied to Machine1.
> >
> > Thanks
> > mark
> >
> > "Pegasus [MVP]" wrote:
> >
> >>
> >>
> >> "Mark" <> said this in news item
> >> news:82165B35-9D63-4D84-841C-...
> >> > Hi...
> >> >
> >> > I have a batch file I'm launching from a web page using psexec.exe.
> >> > The
> >> > last stage of the batch file is to copy a file to several locations.
> >> > The
> >> > problem is that 2 of the 3 destinations are going through a network
> >> > pipe
> >> > the
> >> > size of a swizzle stick.
> >> >
> >> > I was hoping to cut down on the pain by parallelizing the copies with
> >> > START
> >> > (i.e. move the actual XCOPY command into its own little bat and have
> >> > the
> >> > parent bat start them all at once).
> >> >
> >> > I figured that the bottleneck is the network pipe, not the disk access
> >> > to
> >> > the local file being copied so I might save some time by getting all
> >> > the
> >> > copies going at once.
> >> >
> >> > Problem is that when I ran this up the flagpole, the parent bat just
> >> > hung
> >> > at
> >> > the start command. Didn't copy anything. Is START something you
> >> > shouldn't
> >> > use when you're already in the background? Or running under psexec?
> >> >
> >> > Thanks
> >> > Mark
> >>
> >> This is probably easy to work out but my crystal ball is a little cloudy
> >> right now. I've booked it for a cut & polish service early next week.
> >> Care
> >> to post your batch file in the meantime?
> >>

 
Reply With Quote
 
Pegasus [MVP]
Guest
Posts: n/a

 
      02-01-2010
Here are a couple of pointers:
- Terminate each of your Copy.MachineX.bat files with an explicit "exit"
command.
- The syntax for the Start command is:
START ["Title"] [/D Path] [/I] [/MIN] [/MAX] [/SEPARATE | /SHARED] . . .
The "Title" parameter is compulsory when the program name you invoke has
embedded spaces and is surrounded by double quotes.


"Mark" <> said this in news item
news:5B5A4787-5D5C-4845-922E-...
> The output was already being captured farther up. It gets as far as the
> start command, where it hangs. Nothing in the bat to be executed comes
> out,
> nothing after the first start command.
>
> "Pegasus [MVP]" wrote:
>
>> To find out where your batch file gets stuck, use the syntax below, then
>> examine the log file c:\test.txt. It probably tells you everything you
>> need
>> to know.
>>
>> @echo off
>> ECHO Copy call for %2 on %date% at %time% (User=%UserName%) >>
>> c:\Test.txt
>> IF NOT DEFINED StageServer GOTO :eof
>> start /b /high c:\Tools\Copy.Machine1.bat %1 %2
>> start /b /high c:\Tools\Copy.Machine2.bat %1 %2
>> start /b /high /wait cmd /c c:\tools\Copy.Machine3.bat %1 %2
>> ECHO Copy call for %2 on %date% at %time% >> c:\Test.txt
>>
>> @echo off
>> ECHO Starting Machine001 copy on %date% at %time% >> c:\Test.txt
>> NET USE \\Machine1\ /user:Machine1\%username% 1>> c:\Test.txt 2>>&1
>> XCOPY /Y /c "%1" "\\Machine1\%2" 1>> c:\Test.txt 2>>&1
>> ECHO Done with Machine1 copy on %date% at %time% >> c:\Test.txt
>>
>>
>> "Mark" <> said this in news item
>> news:2570F589-CC65-42F9-921C-...
>> > The whole system is too large to post here, but the part I'm asking
>> > about
>> > is
>> > a bat file spawned from a javascript .wsf file:
>> > ECHO Copy call for %2
>> > time /t
>> > @IF NOT DEFINED StageServer GOTO END
>> > start /b /high Copy.Machine1.bat %1 %2
>> > start /b /high Copy.Machine2.bat %1 %2
>> > start /b /high /wait cmd /c Copy.Machine3.bat %1 %2
>> > time /t
>> > :END
>> >
>> > And each of the Copy bat files looks like this:
>> > @ECHO Starting Machine001 copy
>> > time/t
>> > @NET USE \\Machine1\ /user:Machine1\%username%
>> > XCOPY /Y "%1" "\\Machine1\%2"
>> > @ECHO Done with Machine1 copy
>> > time/t
>> >
>> > Using Procexp.exe, I can see that the bat file is hung on the first
>> > start
>> > command and never finishes. Nor is any file copied to Machine1.
>> >
>> > Thanks
>> > mark
>> >
>> > "Pegasus [MVP]" wrote:
>> >
>> >>
>> >>
>> >> "Mark" <> said this in news item
>> >> news:82165B35-9D63-4D84-841C-...
>> >> > Hi...
>> >> >
>> >> > I have a batch file I'm launching from a web page using psexec.exe.
>> >> > The
>> >> > last stage of the batch file is to copy a file to several locations.
>> >> > The
>> >> > problem is that 2 of the 3 destinations are going through a network
>> >> > pipe
>> >> > the
>> >> > size of a swizzle stick.
>> >> >
>> >> > I was hoping to cut down on the pain by parallelizing the copies
>> >> > with
>> >> > START
>> >> > (i.e. move the actual XCOPY command into its own little bat and have
>> >> > the
>> >> > parent bat start them all at once).
>> >> >
>> >> > I figured that the bottleneck is the network pipe, not the disk
>> >> > access
>> >> > to
>> >> > the local file being copied so I might save some time by getting all
>> >> > the
>> >> > copies going at once.
>> >> >
>> >> > Problem is that when I ran this up the flagpole, the parent bat just
>> >> > hung
>> >> > at
>> >> > the start command. Didn't copy anything. Is START something you
>> >> > shouldn't
>> >> > use when you're already in the background? Or running under psexec?
>> >> >
>> >> > Thanks
>> >> > Mark
>> >>
>> >> This is probably easy to work out but my crystal ball is a little
>> >> cloudy
>> >> right now. I've booked it for a cut & polish service early next week.
>> >> Care
>> >> to post your batch file in the meantime?
>> >>

 
Reply With Quote
 
Mark
Guest
Posts: n/a

 
      02-02-2010

Not sure which one actually did the trick, but I added

start "title" /b cmd /s /c C:\tools\Copy,machine1.bat %1 %2

(adding the title and cmd /s /c in lieu of exit in the bat file)

And it worked!

Thanks for your help...
Mark


"Pegasus [MVP]" wrote:

> Here are a couple of pointers:
> - Terminate each of your Copy.MachineX.bat files with an explicit "exit"
> command.
> - The syntax for the Start command is:
> START ["Title"] [/D Path] [/I] [/MIN] [/MAX] [/SEPARATE | /SHARED] . . .
> The "Title" parameter is compulsory when the program name you invoke has
> embedded spaces and is surrounded by double quotes.
>
>
> "Mark" <> said this in news item
> news:5B5A4787-5D5C-4845-922E-...
> > The output was already being captured farther up. It gets as far as the
> > start command, where it hangs. Nothing in the bat to be executed comes
> > out,
> > nothing after the first start command.
> >
> > "Pegasus [MVP]" wrote:
> >
> >> To find out where your batch file gets stuck, use the syntax below, then
> >> examine the log file c:\test.txt. It probably tells you everything you
> >> need
> >> to know.
> >>
> >> @echo off
> >> ECHO Copy call for %2 on %date% at %time% (User=%UserName%) >>
> >> c:\Test.txt
> >> IF NOT DEFINED StageServer GOTO :eof
> >> start /b /high c:\Tools\Copy.Machine1.bat %1 %2
> >> start /b /high c:\Tools\Copy.Machine2.bat %1 %2
> >> start /b /high /wait cmd /c c:\tools\Copy.Machine3.bat %1 %2
> >> ECHO Copy call for %2 on %date% at %time% >> c:\Test.txt
> >>
> >> @echo off
> >> ECHO Starting Machine001 copy on %date% at %time% >> c:\Test.txt
> >> NET USE \\Machine1\ /user:Machine1\%username% 1>> c:\Test.txt 2>>&1
> >> XCOPY /Y /c "%1" "\\Machine1\%2" 1>> c:\Test.txt 2>>&1
> >> ECHO Done with Machine1 copy on %date% at %time% >> c:\Test.txt
> >>
> >>
> >> "Mark" <> said this in news item
> >> news:2570F589-CC65-42F9-921C-...
> >> > The whole system is too large to post here, but the part I'm asking
> >> > about
> >> > is
> >> > a bat file spawned from a javascript .wsf file:
> >> > ECHO Copy call for %2
> >> > time /t
> >> > @IF NOT DEFINED StageServer GOTO END
> >> > start /b /high Copy.Machine1.bat %1 %2
> >> > start /b /high Copy.Machine2.bat %1 %2
> >> > start /b /high /wait cmd /c Copy.Machine3.bat %1 %2
> >> > time /t
> >> > :END
> >> >
> >> > And each of the Copy bat files looks like this:
> >> > @ECHO Starting Machine001 copy
> >> > time/t
> >> > @NET USE \\Machine1\ /user:Machine1\%username%
> >> > XCOPY /Y "%1" "\\Machine1\%2"
> >> > @ECHO Done with Machine1 copy
> >> > time/t
> >> >
> >> > Using Procexp.exe, I can see that the bat file is hung on the first
> >> > start
> >> > command and never finishes. Nor is any file copied to Machine1.
> >> >
> >> > Thanks
> >> > mark
> >> >
> >> > "Pegasus [MVP]" wrote:
> >> >
> >> >>
> >> >>
> >> >> "Mark" <> said this in news item
> >> >> news:82165B35-9D63-4D84-841C-...
> >> >> > Hi...
> >> >> >
> >> >> > I have a batch file I'm launching from a web page using psexec.exe.
> >> >> > The
> >> >> > last stage of the batch file is to copy a file to several locations.
> >> >> > The
> >> >> > problem is that 2 of the 3 destinations are going through a network
> >> >> > pipe
> >> >> > the
> >> >> > size of a swizzle stick.
> >> >> >
> >> >> > I was hoping to cut down on the pain by parallelizing the copies
> >> >> > with
> >> >> > START
> >> >> > (i.e. move the actual XCOPY command into its own little bat and have
> >> >> > the
> >> >> > parent bat start them all at once).
> >> >> >
> >> >> > I figured that the bottleneck is the network pipe, not the disk
> >> >> > access
> >> >> > to
> >> >> > the local file being copied so I might save some time by getting all
> >> >> > the
> >> >> > copies going at once.
> >> >> >
> >> >> > Problem is that when I ran this up the flagpole, the parent bat just
> >> >> > hung
> >> >> > at
> >> >> > the start command. Didn't copy anything. Is START something you
> >> >> > shouldn't
> >> >> > use when you're already in the background? Or running under psexec?
> >> >> >
> >> >> > Thanks
> >> >> > Mark
> >> >>
> >> >> This is probably easy to work out but my crystal ball is a little
> >> >> cloudy
> >> >> right now. I've booked it for a cut & polish service early next week.
> >> >> Care
> >> >> to post your batch file in the meantime?
> >> >>

 
Reply With Quote
 
Pegasus [MVP]
Guest
Posts: n/a

 
      02-03-2010


"Mark" <> said this in news item
news:0127AE29-F415-46F6-9B1E-...
> Not sure which one actually did the trick, but I added
>
> start "title" /b cmd /s /c C:\tools\Copy,machine1.bat %1 %2
>
> (adding the title and cmd /s /c in lieu of exit in the bat file)
>
> And it worked!
>
> Thanks for your help...
> Mark
>


Thanks for the feedback.

 
Reply With Quote
 
 
 
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off




1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59