Jenkins Shared Libraries in Plastic SCM

[Mikael Kalms]

Hi,

I am about to move out some shared logic from our Jenkinsfiles to shared libraries, but I am running into problems.

 

The Plastic SCM integration for Jenkins will always place content into a workspace folder beneath Jenkins' "base" project folder. This is different to how the Jenkins support for other SCMs (Git, Subversion, etc) works.

This is fine for projects that are to be built. However, it seems that I cannot use Plastic SCM for shared libraries: Jenkins expects the src/ and vars/ folders to be directly in the "base" project folder, but with Plastic they will typically end up in the "base"/Jenkins-${JOB_NAME}-${NODE_NAME} folder.

To be more concrete, on my Jenkins machine the shared library files will be checked out at 

/var/lib/jenkins/workspace/Tests/TestSharedLibrary@libs/FreedomJenkinsLibrary/Jenkins-TestSharedLibrary-MASTER

but Jenkins requires the repository's files to exist at

/var/lib/jenkins/workspace/Tests/TestSharedLibrary@libs/FreedomJenkinsLibrary

The error from Jenkins is:

ERROR: Library FreedomJenkinsLibrary expected to contain at least one of src or vars directories

 

Do you have any solution for this? Otherwise, I may have to resort to putting the shared library code into GitHub, and that would make our operational processes more complicated.

 

Corresponding JIRA in Jenkins CI: https://issues.jenkins-ci.org/browse/JENKINS-50284

[ruben]

Hi Mikael,

We changed that behavior long time ago to support multiple plastic workspaces for the same jenkins project (The  'additional workspaces' entry in the Source Code Management section).

This way you are able to setup different workspaces for the same jenkins project and customize their relative paths through the 'Workspace Name' field.

Couldn't you configure your project builds to take advantage of this behavior in your setup?

If it doesn't suit you at all, we could review how to support also plastic workspaces directly in the jenkins workspace.

[Mikael Kalms]

Ideally I would like to:

* Have one single (large) repository with the source code for my product; this does not necessarily have to include the build configurations

* Describe my build configurations using Declarative Pipeline syntax, and have this stored somewhere in source control

* Be able to share common build logic between multiple build configurations, and have this shared logic also stored somewhere in source control

 

Today I have a single Plastic repo that contains the product + 5 build configuration files (that is, 5 Jenkinsfiles). I have one Jenkins job configured for each build configuration, and I specify the appropriate Jenkinsfile in the Pipeline Job configuration. This is reasonable... except that I have a blob of Groovy support-functions at the top of each Jenkinsfile. I would like to be able to move out that blob of support functions to a couple of Groovy source files in a single, shared location.

If I was using Git, I would move the Groovy support-functions to a separate Git repository. I would add that Git repository to Jenkins list of Global Shared Libraries. Then I would add a "@Library("my-support-library") _" line at the top of each Jenkinsfile. This would work - at least on paper.

 

Is there any solution that allows me to share the logic if files are kept in Plastic repo(s)? When I Google on this most results say "use Shared Libraries".

(Pushing out the build logic from Groovy to a different language - batch files, Python scripts, etc - will make it trivial to share the batch/Python/etc source files between the different build configurations, but then I lose access to all the existing Jenkins plugins and internal Jenkins state.)

[Mikael Kalms]

I am experimenting a bit with loading in the support modules using the 'load' command in Groovy.

Problem: on the master, the "Pipeline script from SCM" will result in the repository being checked out to .../TestProject@script/PlasticWorkspaceName/<files> but when the Jenkinsfile is executed, the context (both ${WORKSPACE} and pwd()) point to the .../TestProject/, i.e. the root of the 'real' workspace checkouts on the slaves.  I do not know how to phrase the load statement correctly on both master and slave.

[psantosl]

Hi Mikael,

 

We are currently working on reviewing all our major CI integrations, and Jenkins is a top priority.

 

We would like to dig deeper on the workflow you propose.

 

Would you mind reaching me at pablo at codicesoftware.com to schedule an online session as soon as possible and discuss this topic?

 

Thanks for your support!

 

pablo

[psantosl]

We had a short meeting with Mikael today.

* We need to improve integration with BlueOcean. We'll do.

* We are going to create an alternative for the "create workspaces under a fixed directory" to support shared libraries in Jenkins

[ruben]

Hi Mikael,

We have just release the version 2.14  of the Plastic SCM - Jenkins plugin.

Now, you have a checkbox in the 'source code management' configuration to use multiple workspaces or not. If it isn't checked, the plastic workspace path & the jenkins workspace path will match fixing (hopefully) any issue with the shared libraries.

Thank you for your detailed report!

Rubén.

[Mikael Kalms]

Hi,

I am revisiting the shared library situation. The ability to not use multiple workspaces helps (thanks!), but I have run into another blocker: Shared libraries can only be used from one Jenkins project at a time.

 

I haven't dug through the Plastic plugin Java code yet, but I suspect that it is a Plastic plugin <-> Jenkins incompatibility. My guess:

1. Jenkins wants to check out the shared library once for each job that uses it (to folders with names like "<job>@libs/<sharedlibrary>")

2. The Plastic plugin will create a workspace when checking out a repository, regardless of whether it's a job or a shared library that needs to be checked out.

3. The Plastic plugin will use a workspace name on the form "<job>#<node>" or "<sharedlibrary>#<node>"

1+2+3 => If the same shared library is used by two jobs, the Plastic plugin will attempt to use the same workspace name for two different workspaces on the same node. This will not work.

 

I'm not sure how to solve this. If you have enough context in the plugin to know when Jenkins attempts to check out a shared library as part of running a job, you might be able to generate workspace names like "<job>#<sharedlibrary>#<node>". Otherwise, perhaps the answer is to move to encoding a larger part of the raw path into the workspace name?

 

Mikael

------------------------

 

Repro case:

Let's say I have one project with the Shared Library code in. That project is called JenkinsBuildScripts.

I have two other projects which will use the Shared Library code. These projects are called JenkinsBuildScripts-Example and JenkinsBuildScripts-Example2.

This is the entire Jenkinsfile for each of the example projects:

@Library("JenkinsBuildScripts") _

pipeline {
	agent { label 'master' }

	stages {
		stage('Example stage') {
			steps {
				echo "Stage is running"
			}
		}
	}
}

So far, nothing strange.

Now, when I run the job for JenkinsBuildScripts-Example I get the output below. Plastic will locate the JenkinsBuildScripts workspace (creating a new one if it does not exist), refresh that, then proceed to building stuff, and all is good:

Started by timer
Obtained Jenkinsfile.groovy from repository "JenkinsBuildScripts-Example@<organization redacted>@cloud"
  path "/"
    smartbranch "/main"
Running in Durability level: MAX_SURVIVABILITY
Loading library JenkinsBuildScripts@latest
[JenkinsBuildScripts] $ cm find changeset where date between '2018-12-23T14:03:00' and '2018-12-23T14:05:00' and branch='/main' on repositories 'JenkinsBuildScripts@<organization redacted>@Cloud' --xml --dateformat=yyyy'-'MM'-'dd'T'HH':'mm':'ss
<?xml version="1.0" encoding="utf-8" ?>
<PLASTICQUERY>
</PLASTICQUERY>
[JenkinsBuildScripts] $ cm gwp /var/lib/jenkins/workspace/JenkinsBuildScripts-Example@libs/JenkinsBuildScripts --format={1}
/var/lib/jenkins/workspace/JenkinsBuildScripts-Example@libs/JenkinsBuildScripts
[JenkinsBuildScripts] $ cm lwk --format={0}#{1}#{2}
...
JenkinsBuildScripts#master#/var/lib/jenkins/workspace/JenkinsBuildScripts-Example@libs/JenkinsBuildScripts
JenkinsBuildScripts-Example#master#/var/lib/jenkins/workspace/JenkinsBuildScripts-Example
...
[JenkinsBuildScripts] $ cm unco --all /var/lib/jenkins/workspace/JenkinsBuildScripts-Example@libs/JenkinsBuildScripts
[JenkinsBuildScripts] $ cm ss wk:JenkinsBuildScripts
Selector for workspace JenkinsBuildScripts:
repository "JenkinsBuildScripts@<organization redacted>@Cloud"
  path "/"
    smartbranch "/main"

[JenkinsBuildScripts] $ cm update /var/lib/jenkins/workspace/JenkinsBuildScripts-Example@libs/JenkinsBuildScripts
The workspace /var/lib/jenkins/workspace/JenkinsBuildScripts-Example@libs/JenkinsBuildScripts is up-to-date (cset:43@JenkinsBuildScripts@<organization redacted>@Cloud)
[Pipeline] node
Running on Jenkins in /var/lib/jenkins/workspace/JenkinsBuildScripts-Example
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Declarative: Checkout SCM)
[Pipeline] checkout
[JenkinsBuildScripts-Example] $ cm find changeset where date between '2018-12-23T14:03:00' and '2018-12-23T14:05:00' and branch='/main' on repositories 'JenkinsBuildScripts-Example@<organization redacted>@cloud' --xml --dateformat=yyyy'-'MM'-'dd'T'HH':'mm':'ss
<?xml version="1.0" encoding="utf-8" ?>
<PLASTICQUERY>
</PLASTICQUERY>
[JenkinsBuildScripts-Example] $ cm gwp /var/lib/jenkins/workspace/JenkinsBuildScripts-Example --format={1}
/var/lib/jenkins/workspace/JenkinsBuildScripts-Example
[JenkinsBuildScripts-Example] $ cm lwk --format={0}#{1}#{2}
...
JenkinsBuildScripts#master#/var/lib/jenkins/workspace/JenkinsBuildScripts-Example@libs/JenkinsBuildScripts
JenkinsBuildScripts-Example#master#/var/lib/jenkins/workspace/JenkinsBuildScripts-Example
...
[JenkinsBuildScripts-Example] $ cm unco --all /var/lib/jenkins/workspace/JenkinsBuildScripts-Example
[JenkinsBuildScripts-Example] $ cm ss wk:JenkinsBuildScripts-Example
Selector for workspace JenkinsBuildScripts-Example:
repository "JenkinsBuildScripts-Example@<organization redacted>@cloud"
  path "/"
    smartbranch "/main"

[JenkinsBuildScripts-Example] $ cm update /var/lib/jenkins/workspace/JenkinsBuildScripts-Example
The workspace /var/lib/jenkins/workspace/JenkinsBuildScripts-Example is up-to-date (cset:2@JenkinsBuildScripts-Example@<organization redacted>@cloud)
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Example stage)
[Pipeline] echo
Stage has begun
[Pipeline] script
[Pipeline] {
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
...

However, when I run the job for JenkinsBuildScripts-Example2, I get the output below. Plastic will check whether or not there is a workspace for JenkinsBuildScripts under the JenkinsBuildScripts-Example2@libs folder,  find no workspace there, then skip creation of a new workspace, then attempt to revert any changes in the not-yet-created workspace, and this will fail:

Obtained Jenkinsfile.groovy from repository "JenkinsBuildScripts-Example@<organization redacted>@Cloud"
  path "/"
    smartbranch "/main"
Running in Durability level: MAX_SURVIVABILITY
Loading library JenkinsBuildScripts@latest
[JenkinsBuildScripts] $ cm find changeset where date between '2018-12-23T13:57:05' and '2018-12-23T14:11:18' and branch='/main' on repositories 'JenkinsBuildScripts@<organization redacted>@Cloud' --xml --dateformat=yyyy'-'MM'-'dd'T'HH':'mm':'ss
<?xml version="1.0" encoding="utf-8" ?>
<PLASTICQUERY>
</PLASTICQUERY>
[JenkinsBuildScripts] $ cm gwp /var/lib/jenkins/workspace/JenkinsBuildScripts-Example2@libs/JenkinsBuildScripts --format={1}
/var/lib/jenkins/workspace/JenkinsBuildScripts-Example2@libs/JenkinsBuildScripts is not in a workspace.
[JenkinsBuildScripts] $ cm lwk --format={0}#{1}#{2}
...
JenkinsBuildScripts#master#/var/lib/jenkins/workspace/JenkinsBuildScripts-Example@libs/JenkinsBuildScripts
JenkinsBuildScripts-Example#master#/var/lib/jenkins/workspace/JenkinsBuildScripts-Example
...
[JenkinsBuildScripts] $ cm unco --all /var/lib/jenkins/workspace/JenkinsBuildScripts-Example2@libs/JenkinsBuildScripts
/var/lib/jenkins/workspace/JenkinsBuildScripts-Example2@libs/JenkinsBuildScripts is not in a workspace.
[JenkinsBuildScripts] $ cm unco --all /var/lib/jenkins/workspace/JenkinsBuildScripts-Example2@libs/JenkinsBuildScripts
/var/lib/jenkins/workspace/JenkinsBuildScripts-Example2@libs/JenkinsBuildScripts is not in a workspace.
[JenkinsBuildScripts] $ cm unco --all /var/lib/jenkins/workspace/JenkinsBuildScripts-Example2@libs/JenkinsBuildScripts
/var/lib/jenkins/workspace/JenkinsBuildScripts-Example2@libs/JenkinsBuildScripts is not in a workspace.
FATAL: The cm command 'cm unco --all /var/lib/jenkins/workspace/JenkinsBuildScripts-Example2@libs/JenkinsBuildScripts' failed after 3 retries
FATAL: The cm command 'cm unco --all /var/lib/jenkins/workspace/JenkinsBuildScripts-Example2@libs/JenkinsBuildScripts' failed after 3 retries
ERROR: Maximum checkout retry attempts reached, aborting
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: Loading libraries failed

1 error

	at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
	at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085)
	at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
	at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
	at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
	at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
	at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
	at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
	at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
	at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:131)
	at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:125)
	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:560)
	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:521)
	at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:319)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
Finished: FAILURE

 

[calbzam]

Hi,

- How are you defining the shared libraries in the Jenkins configuration? Could you attach some screenshots of your build settings? Reading some documentation, I can see you can define Global shared Libraries following these steps: https://jenkins.io/doc/book/pipeline/shared-libraries/ 

Is this what you are configuring, right?

- Why don't you use Plastic Xlinks for your shared libraries/repos? Doesn't it help as a workaround?

Regards,

Carlos.

[Mikael Kalms]

Correct, I am declaring a Global Shared Library in the settings. Here are the settings that I use (see screenshot, sorry for low resolution, taken on phone). 

You should be able to repro this by setting up an empty Plastic repo as your Global Shared Library, and another Plastic repo containing a single-line Jenkinsfile:

    @Library('JenkinsBuildScripts@latest') _

 

Then you would set up two Jenkins jobs referencing the latter repo. The first to build will be succrssful in downloading the global shared library, the second will not. Let me know if you want a tested example from me. 

 

As for 'why not Xlinks' - the global shared libs are more than just sub-repos, there is also a set of Initialization rules associated with loading a library. The Jenkins security boundary is also different between the library and the code using the boundary.

 

For the time being I'll work around these problems by putting the Global Shared Library code into a Git repo. 

 

[calbzam]

Hi,

I was able to reproduce your scenario. Let me share with the team to check if we can find a workaround or we can improve the plugin to support it.

Sorry for the inconveniences,

Carlos.

[omris]

Hi,

we're trying to use shared library and faced the same problem described above. Any update? We're using the latest PlasticSCM server(on prem) and Jenkins plugin version

[calbzam]

Hi, I'm afraid the task is not yet scheduled. Sorry for the inconveniences. 

Carlos.

[Lars]

Hi all,

we have the same issue. Any updates?

A work around might be the usage of the git server for plastic. Using this I have trouble so set up the default version though. 

Can you help me with this?

All the best

Lars

[Lars]

Never mind. It works now using the version "master"

Never the less a direct solution using the plastic plugin would be preferable since the git server would then be unessesary

[calbzam]

Hi, I will try to push the task but I'm afraid it's not yet scheduled. Good to know that you at least found a workaround via GitServer.

Regards,

Carlos.

[Mikael Kalms]

v2.22 of the PlasticSCM Jenkins plugin includes the fix. Thanks! See JIRA issue JENKINS-50284. Confirmed that this works for us: we are again using Plastic for hosting our Jenkins Shared Library.

[calbzam]

Great! Thanks for the update

[Evgeny]

I think, i run into the same problem with Templating Engine plugin for Jenkins. https://plugins.jenkins.io/templating-engine/

In "Configure System" you can set 2 repositories:

1) Pipeline Configuration - works fine with Plastic
2) Library Sources - don't work with plastic, but works ok with GIT.

Any ideas?

[calbzam]

Hi,

- After our last fix, the plugin should support using Shared Libraries. Are you using the last version, right?

https://issues.jenkins-ci.org/browse/JENKINS-50284?focusedCommentId=370126&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel

-  I guess the current issue is not with the support for Shared Libraries but when this "Templating Engine" plugin is trying to use it? We can give it a try as we are not familiar with this plugin.

Regards,

Carlos.

[Evgeny]

I'm not using shared libraries. This plugin has his analogy of shared libraries.

I'm using PlasticSCM Plugin 3.3

[calbzam]

Could you open a thread also with this plugin mantainer? We can double check that the Jenkins shared libraries are still supported but not sure if we can do something on our side to fix/improve this third party plugin so it also works with this analogy of the shared libraries.

Regards,

Carlos.

[Evgeny]

Never mind, i decided not to use this plugin.

Shared libraries work fine with Plastic!