Jump to content

Problems with TeamCity plugin


CodingGorilla

Recommended Posts

I'm having a lot of problems with the new version of the TeamCity plugin. I was able to get it to make the initial build (right after I attached the VCS root). But I have not been able to get it to make subsequent builds. The most recent attempt gave me the following error:

Failed for the root 'repository "ThreeSixty"

path "/"

smartbranch "/main"' #35: Error getting log from cs:86@rep:ThreeSixty@repserver:dev-storage:8087 to cs:92@rep:ThreeSixty@repserver:dev-storage:8087: Error: Object reference not set to an instance of an object.

Here's the stack trace:

jetbrains.buildServer.vcs.VcsException: Error getting log from cs:86@rep:ThreeSixty@repserver:dev-storage:8087 to cs:92@rep:ThreeSixty@repserver:dev-storage:8087: Error: Object reference not set to an instance of an object.

at com.codicesoftware.plugins.teamcity.PlasticVcsSupport.buildPatch(PlasticVcsSupport.java:283)

at jetbrains.buildServer.serverSide.impl.projectSources.BuildPatchUtil.buildPatch(BuildPatchUtil.java:6)

at jetbrains.buildServer.serverSide.impl.projectSources.PatchComposer.makePatch(PatchComposer.java:20)

at jetbrains.buildServer.serverSide.impl.projectSources.PatchComposer.buildPatchForRoot(PatchComposer.java:83)

at jetbrains.buildServer.serverSide.impl.projectSources.PatchComposer.buildPatch(PatchComposer.java:2)

at jetbrains.buildServer.serverSide.impl.BuildTypeImpl.buildPatch(BuildTypeImpl.java:245)

at jetbrains.buildServer.serverSide.impl.BuildTypeImpl$$FastClassByCGLIB$$a84db719.invoke(<generated>)

at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:149)

at org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:700)

at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)

at jetbrains.buildServer.serverSide.impl.auth.TeamCityMethodSecurityInterceptor.invoke(TeamCityMethodSecurityInterceptor.java:7)

at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)

at org.springframework.aop.framework.Cglib2AopProxy$FixedChainStaticTargetInterceptor.intercept(Cglib2AopProxy.java:582)

at jetbrains.buildServer.serverSide.impl.BuildTypeImpl$$EnhancerByCGLIB$$ba35214f.buildPatch(<generated>)

at jetbrains.buildServer.serverSide.impl.BuildStarter$2.call(BuildStarter.java:2)

at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)

at java.util.concurrent.FutureTask.run(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

at java.lang.Thread.run(Unknown Source)

Caused by: com.codicesoftware.plastic.core.PlasticException: Error: Object reference not set to an instance of an object.

at com.codicesoftware.plastic.core.PlasticShell.execute(Unknown Source)

at com.codicesoftware.plastic.commands.LogCommand.execute(Unknown Source)

at com.codicesoftware.plugins.teamcity.PlasticVcsSupport.buildPatch(PlasticVcsSupport.java:279)

... 19 more

com.codicesoftware.plastic.core.PlasticException: Error: Object reference not set to an instance of an object.

at com.codicesoftware.plastic.core.PlasticShell.execute(Unknown Source)

at com.codicesoftware.plastic.commands.LogCommand.execute(Unknown Source)

at com.codicesoftware.plugins.teamcity.PlasticVcsSupport.buildPatch(PlasticVcsSupport.java:279)

at jetbrains.buildServer.serverSide.impl.projectSources.BuildPatchUtil.buildPatch(BuildPatchUtil.java:6)

at jetbrains.buildServer.serverSide.impl.projectSources.PatchComposer.makePatch(PatchComposer.java:20)

at jetbrains.buildServer.serverSide.impl.projectSources.PatchComposer.buildPatchForRoot(PatchComposer.java:83)

at jetbrains.buildServer.serverSide.impl.projectSources.PatchComposer.buildPatch(PatchComposer.java:2)

at jetbrains.buildServer.serverSide.impl.BuildTypeImpl.buildPatch(BuildTypeImpl.java:245)

at jetbrains.buildServer.serverSide.impl.BuildTypeImpl$$FastClassByCGLIB$$a84db719.invoke(<generated>)

at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:149)

at org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:700)

at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)

at jetbrains.buildServer.serverSide.impl.auth.TeamCityMethodSecurityInterceptor.invoke(TeamCityMethodSecurityInterceptor.java:7)

at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)

at org.springframework.aop.framework.Cglib2AopProxy$FixedChainStaticTargetInterceptor.intercept(Cglib2AopProxy.java:582)

at jetbrains.buildServer.serverSide.impl.BuildTypeImpl$$EnhancerByCGLIB$$ba35214f.buildPatch(<generated>)

at jetbrains.buildServer.serverSide.impl.BuildStarter$2.call(BuildStarter.java:2)

at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)

at java.util.concurrent.FutureTask.run(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

at java.lang.Thread.run(Unknown Source)

Any ideas?

Link to comment
Share on other sites

  • Replies 90
  • Created
  • Last Reply
  • 3 weeks later...

Got it, and so far so good!

One question, do you know if it's possible and if so how I can get just the changeset number in a TeamCity variable?

Right now in my version Build Number Format I am using: 1.0.%build.vcs.number.Plastic_ThreeSixty%.2-{0}

Which results in: #1.0.cs:136@rep:ThreeSixty@repserver:dev-storage:8087.2-190

Where what I would like is: #1.0.136.2-190

Is it doable?

Link to comment
Share on other sites

I may have spoke (posted) too soon.

It doesn't seem to like doing manual builds (ie. No changes from Plastic occured, I manually kicked off a build). I see "Checking for changes" in the build log and nothing else. When I do a thread dump, it tells me: "TeamCity was unble to locate any processes of this build.". I let it run (so far) for 3m more than the previous build, which completed properly, and nothing.

Even after changing a couple of files and checking them in, TeamCity would not detect the changes and run the build. Stopping the TeamCity Web Server service and restarting it made it pick up the changes and it's actually building now.

Let me know if there's any additional information I can offer.

Additionally, I see the following in: teamcity-javaLogging-2012-01-10.log:

Jan 10, 2012 2:23:34 PM com.codicesoftware.plugins.teamcity.PlasticVcsSupport updateWorkspace

SEVERE: Error getting workspace from path C:\teamcity-config\system\caches\plasticscm\plasticscm_-2364678487989621573:

More info:

It seems like I can't get more than one build at a time. Even if I get a successful build after restarting the service, and then I push some changes, it never seems to detect the new changes and kick off another build. Again, if I stop and restart the "TeamCity Web Service" service then it picks up the changes and functions normally.

I notice that there's a child process of 'tomcat6.exe' (which I'm assuming is teamcity web server) called 'cm.exe' that is always there after the initial build completes. Right after the web server restart, I notice cm.exe starting and stopping a few times (as I would expect if it were executing several commands sequentially). After the build is complete, it just runs and seems to stay there forever.

More info 2:

If I attempt to debug the cm.exe child process with Visual Studio, VS tells me that the process appears to be deadlocked because it is not executing any user mode code. Further, if I kill this process and wait about a minute or so, it fires back up and TeamCity starts noticing my changes again and kicking off builds.

More info 3:

It seems like it's just that first time after a build right after the service starts. I've had 3 in a row run properly once I killed that hanging cm.exe (without restarting the web server service). In Process Explorer I now see cm.exe starting and stopping at regular intervals (60 second check interval I would assume) like I would expect. I will try stopping and restarting the TeamCity Web Server service and running another build to see if I can reproduce all of this.

Link to comment
Share on other sites

Hi CodingGorilla,

I've just installed a TeamCity (TeamCity Professional 6.5.6 (build 18130)) in my machine (Windows 7 64bits) and everything is working fine.

Can you show us your TeamCity Project setting in order to see if you have something different from us?

One question, if you don't trigger the manual build is the TeamCity working fine with new changes in the repository?

Link to comment
Share on other sites

The server did some updates and rebooted last night; so I will try some more testing later today (hopefully). Let me ask you this though; should I be seeing a contiuous 'cm.exe' process? This is a child of the tomcat6.exe process, and it looks like is been running since last night when the server was rebooted. As far as I can tell it's not doing anything; Process Explorer's Thread's list doesn't show anything that looks like it might be "user code". It's all ntdll.dll or mscorwks.dll, so it seems to me this is a dead process.

Any thoughts?

Link to comment
Share on other sites

Ok, well I think this is where the problem is coming from. According to Process Explorer, this process was started 5:03:23pm 1/10/2012 (it's now 11:56am 1/11/2012). There's nothing really special about my VCS settings, here's a screenshot of my VCS Root settings for this project (there is only one TeamCity project using Plastic right now).

http://screencast.com/t/4dKarE7P9G

This is a screenshot of my Build Trigger (VCS Trigger):

http://screencast.com/t/uuuUuR7WuW

This is a screenshot of the Plugin page from TeamCity:

http://screencast.com/t/9W5iwvzZ6F

Are there other settings you would like to know about?

Link to comment
Share on other sites

mmmm no, I have the same settings.

Ok, well I think this is where the problem is coming from. According to Process Explorer, this process was started 5:03:23pm 1/10/2012 (it's now 1:56am 1/11/2012)

hehe, more ideas?

Please, enable the cm log and let's try to find where is stuck.

Link to comment
Share on other sites

I'm seeing a lot of these in the teamcity-vcs.log file:

[2012-01-11 15:16:12,559] DEBUG [cutor 2 {id=35}] - jetbrains.buildServer.VCS - Error getting workspace from path C:\teamcity-config\system\caches\plasticscm\plasticscm_-2364678487989621573:

jetbrains.buildServer.vcs.VcsException: Error getting workspace from path C:\teamcity-config\system\caches\plasticscm\plasticscm_-2364678487989621573:

at com.codicesoftware.plugins.teamcity.PlasticVcsSupport.updateWorkspace(PlasticVcsSupport.java:601)

at com.codicesoftware.plugins.teamcity.PlasticVcsSupport.getCurrentVersion(PlasticVcsSupport.java:472)

at jetbrains.buildServer.vcs.impl.VcsRootInstancesManagerImpl$SVcsRootInstance.getCurrentRevision(VcsRootInstancesManagerImpl.java:5)

at jetbrains.buildServer.vcs.impl.VcsManagerImpl.getRevisionsForAllRoots(VcsManagerImpl.java:444)

at jetbrains.buildServer.vcs.impl.VcsManagerImpl.loadChanges(VcsManagerImpl.java:62)

at jetbrains.buildServer.serverSide.impl.auth.SecuredVcsManager.loadChanges(SecuredVcsManager.java:75)

at jetbrains.buildServer.serverSide.impl.VcsModificationChecker$1.run(VcsModificationChecker.java:0)

at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)

at java.util.concurrent.FutureTask.run(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

at java.lang.Thread.run(Unknown Source)

Caused by: com.codicesoftware.plastic.core.PlasticException:

at com.codicesoftware.plastic.core.PlasticShell.launchShell(Unknown Source)

at com.codicesoftware.plastic.core.PlasticShell.initShell(Unknown Source)

at com.codicesoftware.plastic.core.PlasticShell.execute(Unknown Source)

at com.codicesoftware.plastic.commands.GetWorkspaceFromPathCommand.execute(Unknown Source)

at com.codicesoftware.plugins.teamcity.PlasticVcsSupport.updateWorkspace(PlasticVcsSupport.java:597)

... 12 more

Link to comment
Share on other sites

I think you enable it by adding <StackTrace>YES</StackTrace> to your client.conf file.

I found my client.conf file in C:\Documents and Settings\{USERNAME}\Local Settings\Application Data\plastic4.

This is assuming TeamCity uses the 'cm' command line tool to interface with Plastic similar to how CruiseControl does.

Link to comment
Share on other sites

Did you tried to reset TeamCity's internal workspace cache?

You can clear it via Administration > Server Configuration > reset caches (link on the right side) > hit reset on plasticscm.

This will delete the workspaces and folders that TeamCity uses as server-side checkouts. Wait a few minutes, then rerun a build configuration. Maybe your cached workspaces are "corrupted" and updating them failed.

Regarding the long-running cm.exe process. Seems to be the same issue I had trouble (and still have) with. (). Plastic 4.0.239 and the new TeamCity plugin work for me, but I still had once the problem with the hanging cm.exe. A workaround is to the just kill cm.exe process.

Link to comment
Share on other sites

@Olaf I didn't do it the way you suggested, but yea I've deleted both the cached workspace (as indicated in the error log I posted) as well as the build agent work directory.

To be clear I can get it to build, but it seems like after the first build (after a reboot or service restart) the cm.exe gets hung and future repository checks stop happening. If I kill that dead cm.exe process, it starts working again.

Link to comment
Share on other sites

Ok, let's enable the cm log to know what is happening.

I'm trying to reproduce the same behavior in my machine, so far without luck.

How? What caprediem said?

Update:

Ok, so I did what carpediem said; I have no idea where it will put these logs or stack traces, so someone's going to have to tell me. Although I'm not sure that's going to do much good, the process (cm.exe) doesn't crash, it deadlocks.

But in lieu of further instruction from manu (who only seems to want to talk to me once a day :P) I'll give it a shot.

Link to comment
Share on other sites

To enable the cm log please place this (http://www.codicesoftware.com/externalcontent/KB/Plastic_Logging/cm.log.conf.zip) cm.log.conf file in the same directory where the cm.exe resides. Operations performed by cm.exe will generate a log file (txt file) called cm.log.txt in that directory.

You can also do what carpediem said, it's for giving us the exception stacktrace if there is any.

There is a developer just now working in the issue found by Olaf (http://www.plasticscm.net/index.php?/topic/730-bug-teamcity-plugin-labels-the-wrong-changeset/page__fromsearch__1), she is also going to review the plugin in order to find any strange thing.

Manu.

Link to comment
Share on other sites

Ok, I placed that file in 'C:\Program Files\PlasticSCM4\client', as well as doing what carpediem said. I will have another build ready to run later this afternoon, so I will let you know what happens. I suspect we're not going to get anything more than what I've already provided; like I said the process hangs (deadlocks?) rather than crashing so I don't think we're going to get much. But we'll see.

Thanks for the reply! :D

Link to comment
Share on other sites

Hi CodingGorilla,

the TeamCity plugin uses our java core system and in order to execute a raw of commands it has the capability to open a shell and run all of them faster. You can test it, open a command line and run cm shell, the window will wait for commands and you will be able to start running commands as fast as devil.

So in this plugin is normal to have a permanent cm process running, aside from that you can see also ephemeral cm.exe processes for different purposes.

I've been informed that we have just fixed a bug regarding a cm shell issue with similar symptoms as you are having. I will try to speed up the next release with this fix.

Manu.

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.


×
×
  • Create New...