Client triggers is hanging on checkin.

[Lumz]

Hi,

I've been trying to install a before-clientcheckin trigger and successfully did so until I added a couple of lines to my .bat file.

Since I've added a msbuild.exe command to the script the cm ci command is just hanging forever. So is the GUI if I try to checkin from there. The .bat file is running fine from cmd.exe. I also tried to read the log after setting the .conf file and it seems to get stuck on a cryptic:

2016-12-05 13:56:39,639 DEBUG Channel - Purging connection to plastic://pickledwarf-pc:8087/BranchHandler
2016-12-05 13:56:39,639 DEBUG Channel - Purging connection to plastic://pickledwarf-pc:8087/RepositoryHandler
2016-12-05 13:56:39,639 DEBUG Channel - Purging connection to plastic://pickledwarf-pc:8087/TriggerHandler

Anyone knows what this is about ?

[psantosl]

After 30 seconds of inactivity, network connections are closed.

 

If your bat never finishes, the checkin will wait forever. Chances are your trigger is not finishing as expected.

 

If you want the trigger to be async, then spawn a new process from it and make the trigger return as soon as possible.

[Lumz]

Weird thing is the bat runs without trouble from cmd.exe. No hanging and less than 30 sec.

[Lumz]

If I try to include a CALL msbuild.exe it just hangs there. But not if I run it from cmd.exe.

[manu]

Hello Lumz,

Do you have log inside your script? Do you know if the script starts? 

 

 

[Lumz]

I figured it out.

Plastic doesn't support any ECHO during execution. msbuild.exe was outputting a whole load to stdout and for some reason that was causing the checkin to hang. I added >NUL after msbuild.exe and it solved the problem.

[manu]

Hello Lumz,

I'm sorry but I can't reproduce the issue, I use the following:

echo start
del "C:\Users\manu\Documents\Visual Studio 2015\Projects\ConsoleApplication1\ConsoleApplication1\bin\Debug"\* /F/Q
del "C:\Users\manu\Documents\Visual Studio 2015\Projects\ConsoleApplication1\ConsoleApplication1\obj\Debug"\* /F/Q
C:\Windows\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe "C:\Users\manu\Documents\Visual Studio 2015\Projects\ConsoleApplication1\ConsoleApplication1.sln"
echo end

The checkin operation success and the solution gets compiled.Do you have more info to help us reproduce the issue?

[gweronimo]

Sorry to awaken this old thread, but this is still quite a problem!

I just spent a couple of hours trying to understand why my client-side trigger (before-setselector) would hang often but not always. This trigger called a .bat file that in turn called [cm status --ignored] without redirecting the output to >NUL.

You should be able to reproduce this issue using just a .bat file like this, called from a trigger:

FOR /L %%n IN (0,1,1000) DO  (
  ECHO "Just a lot of stupid text output"
)

For me, this .bat file hangs the trigger, but it no longer does if I decrease the loop number from 1000 to 100. This was tested on the very latest Plastic version (8.0.16.3937).

[calbzam]

Hi,

I'm afraid I cannot reproduce the issue using the same trigger (even increasing the value from 1000 to 10000). Are you just manually running the "cm status --ignored" command?

Do you have multiple ignored items in your workspace?

Regards,

Carlos.

[gweronimo]

Yes we have lots of ignored items, which is how I discovered this problem in the first place. But the [cm status --ignored] command is beside the point, since the issue appears even with a simple echo:ing for-loop. I'm not sure I was clear enough on how to reproduce the issue, so here's a more complete step-by-step description:

Step1 - Create file echo_trigger.bat in workspace root:

@ECHO OFF
FOR /L %%n IN (0,1,1000) DO  (ECHO "Just a lot of stupid text output")

Step 2 - Verify that the echo_trigger.bat script outputs lots of lines when run.

Step 3 - Setup client-side trigger:

> cm trigger create before-setselector "echoing trigger" "@WKSPACE_PATH\echo_trigger.bat"

Step 4 - Run the trigger:

> cm switch br:/main

NOTE: The switch command in step 4 does not terminate, seemingly since the trigger chokes on the amount of output to stdout from the script. (But if I decrease the loop limit from 1000 to 100 and re-run the switch command, it terminates fine.)

Also, if you are really unable to reproduce the hung trigger - please make sure you are actually hitting the trigger, by adding a PAUSE command at the end of the .bat file. Then it should hang the trigger regardless...

Best regards,
Göran W.

[gweronimo]

@calbzam Are you able to reproduce the issue with the above detailed steps?

[calbzam]

Hi again, I'm able to reproduce. Let me share with the team.

Regards,

Carlos.

 

[calbzam]

Hi, 

After debugging the issue in detail:

We are hitting the stdout buffer limit of 4096 bytes because the output is not being sent anywhere.

This can be seen by using this trigger script:

@ECHO OFF
FOR /L %%n IN (0,1,127) DO (ECHO 123456789012345678901234567890)

It writes 32 bytes for each iteration.
127 iterations = 4064 bytes, fine
128 iterations = 4096 bytes -> HANG

Redirecting the output to somewhere "solves" the issue. Would it be a workaround for you?

Best regards,

Carlos.

[Gwyn]

Hi @gweronimo

Please let us know if you are still having problems with your script output, or if Carlos's workaround helped you out.

Thanks,

Gwyn

[gweronimo]

Sorry for delayed answer. The workaround is acceptable, but this problem needs to be documented!

It's very easy to fall into this trap if using some command that does output in the trigger script. It's also very deceptive, since it may work at first and then (even years later) you hit the 4096 mark and BOOM the trigger hangs...

The best solution would be if you could somehow prevent this issue in the Trigger system itself...

Thanks,
Göran W.

[calbzam]

Hi,

We have released the following task:

Quote

 

[Bug] 9.0.16.4292

All platforms - Server: fixed trigger script hangs

If your trigger script produced so much output that it would fill the StandardOutput buffer, the script process would hang.
We fixed that.
Btw, the output buffer is only 4096 bytes.

 

Regards,

Carlos.