Jenkins PlasticSCM Plugin - Exclude certain folder for polling.

[KrisK]

Hello Community!

We in our team embrace the build early and build often mentality. So thats why we build a new version of our game every time a commit occures. The pooling seems to work flawless with the plastic scm jenkins plugin.

Problem: Everytime a build is build the resulting binaries are checked in to a binary folder. Problem: This checkin is triggering a build again which it should not. Thats why I would like to exclude certain folders from polling. Is this possible? If not, what is the work around for this?

[manu]

Hi @KrisK,

the Jenkins plugin is not supporting that for the moment (TeamCity is), but I can give you several alternatives and a good notice for the future.

Alternatives:

* If you have access to the workspace path before the poll (plastic update) takes place you can create a "cloaked.conf" file with the binaries directory on it. That will prevent Plastic from downloading it. More info about the "cloaked.con" file here: https://www.plasticscm.com/documentation/user/plastic-scm-version-control-introduction-guide.shtml?term=cloaked.conf#C2_Flexibility_Tuning

* If you don't have access to the workspace path you can create the "cloaked.conf" at the Plastic SCM client binary location, typically at "C:\Program Files\PlasticSCM5\client" or "/opt/plasticscm5/client". This will apply the cloaked rule for all the workspaces at the Jenkins machine.

* Use a different repo for storing the binaries. This will require you some changes and maybe it's not possible... but if you use a different Plastic SCM repo for storing the new binary you will prevent the new build.

Good notice:

* We are in the middle of a big devops initiative and you will be able (soon) to trigger builds from Plastic so you can decide if you want to build the new changeset or not, we already have it for TeamCity and Bamboo:

 

[KrisK]

Thanks for the quick reply @manu

 

"* If you have access to the workspace path before the poll (plastic update) takes place you can create a "cloaked.conf" file with the binaries directory on it. That will prevent Plastic from downloading it. More info about the "cloaked.con" file here: https://www.plasticscm.com/documentation/user/plastic-scm-version-control-introduction-guide.shtml?term=cloaked.conf#C2_Flexibility_Tuning" -

If I understand you correctly cloaked folders will not be part of polling. So what you suggest is that I have the binary folder cloaked until the build is finished, than I uncloak the binary folder, commit the changes and cloak it again. Is this correct?

Cheers, Kris

[KrisK]

On 5/30/2018 at 10:51 AM, manu said:

* If you have access to the workspace path before the poll (plastic update) takes place you can create a "cloaked.conf" file with the binaries directory on it. That will prevent Plastic from downloading it.

Ok I played around a little and none of your suggestions will work for us. The problem is not the download it is the plastic scm jenkins plugin which triggers the build when content of an folder is  changed even when it is cloaked. I have no possibility to circumvent cyclic builds which is pretty bad keeping in mind that game builds may take around 1 hour to complete. I dont get it all other scm jenkins plugins have the possibility to exclude certain folders from polling :L.

 

On 5/30/2018 at 10:51 AM, manu said:

* Use a different repo for storing the binaries. This will require you some changes and maybe it's not possible.

As you stated this is not possible due to the fact that there is an hard dependency between the binary and the rest.

[manu]

Hi @KrisK,

ok, right now in order to prevent the cyclic build you can do the following, it will require a small check before the build takes place. You have multiple alternatives:

* You can use the Jenkins "PLASTICSCM_CHANGESET_GUID"/"PLASTICSCM_CHANGESET_ID" variables in order to run a diff operation (cm diff), if the diff is returning the binaries directory you can skip the build.

* You can set an attribute to the changeset built (the one that you really need to build as it has real changes), for example CONTENT=CODE and after generating the changeset that contains only new binaries set another attribute to it, for example CONTENT=BIN. Before any Jenkins build you can retrieve the attributes applied to the changeset that is about to be built, it it's Empty or "CODE" then build it, if it's "BIN" skip the build.

* You can use the same strategy as above but using the changeset comment instead of an attribute, set an special comment for the "binary" changeset ("cm ci -c" will allow you to specify a comment), before the build check if the changeset comment is having something like "This aren't the droids you're looking for" and skip the build.

Hope it helps!

[KrisK]

On 6/1/2018 at 11:42 AM, manu said:

* You can use the Jenkins "PLASTICSCM_CHANGESET_GUID"/"PLASTICSCM_CHANGESET_ID" variables in order to run a diff operation (cm diff), if the diff is returning the binaries directory you can skip the build.

How can I access these variables? They are not environment variables as far as I can see, 

echo bat(returnStdout: true, script: 'set')

  is not returning any of the mentioned variables.

Also to get this working properly I need all changesets since the last build. Otherwise I can not iterate through changes and check if something changed since the last build. How to achieve this?

[KrisK]

One more thing:

[BuildXXXX] $ cm find changeset where date between '2018-06-04T12:30:09' and '2018-06-05T12:51:42' and branch='/main' on repositories 'xxxxx:8088' --xml --dateformat=yyyy'-'MM'-'dd'T'HH':'mm':'ss

Is what I can see in the build console output. The output of this command is exactly data I need to identify if we want to continue the build. Where are the given timestamps gathered from?

[manu]

3 hours ago, KrisK said:

How can I access these variables? They are not environment variables as far as I can see, 

As any other Jenkins variable, we added them at the version 2.9 of the plugin:

Version 2.9

From now on, each build will publish environment variables containing the data of the built changeset for each configured workspace. These are the variables exposed by the main workspace of the project:

    PLASTICSCM_CHANGESET_ID: Number of the currently built changeset
    PLASTICSCM_CHANGESET_GUID: GUID of the currently built changeset
    PLASTICSCM_BRANCH: Name of the branch in Plastic SCM
    PLASTICSCM_AUTHOR: Name of the user who created the currently build changeset
    PLASTICSCM_REPSPEC: The configured repository specification for the current build

Additional workspaces will include their position in the list, like this:

    PLASTICSCM_1_CHANGESET_GUID
    PLASTICSCM_5_AUTHOR
    PLASTICSCM_9_CHANGESET_ID
    etc.

 

3 hours ago, KrisK said:

Also to get this working properly I need all changesets since the last build. Otherwise I can not iterate through changes and check if something changed since the last build. How to achieve this?

Not really, I think, maybe I'm missing something from your workflow but if you skip those changesets about to be build that are having a certain attribute set or changeset comment written you will be builing all the right one, correct?

[KrisK]

So my problem was that during a build the repository may change. So I needed a way to identify if someone else except the build machine did change the repository. So here is my solution:

• Get the timestamps for the current and the previous build from a groovy script:
• Use these timestamps to trigger a python script
• The python script uses "cm find revisions where date between 'XXXX' and 'YYYY' and not owner='builduser'  --nototal" to identify if we have any changesets which should trigger a build
• The return code is than used to set an SKIP_BUILD variable which is used as expression in an when directive

Here is a incomplete code example.

        
.....
	stage ("BuildGate"){
            agent {label 'XXX'}
            steps {
                script {
                    print currentBuild.previousBuild.startTimeInMillis;
                    PREV_BUILD_TIMESTAMP = currentBuild.previousBuild.startTimeInMillis;
                    print currentBuild.startTimeInMillis;
                    CUR_BUILD_TIMESTAMP = currentBuild.startTimeInMillis;
                    try {
                        // check if this is a valid build. if this scripts returns non zero exit code catch will be triggered
                        bat "python XXX.py $PREV_BUILD_TIMESTAMP $CUR_BUILD_TIMESTAMP";
                        SKIP_BUILD = false;
                    }
                    catch (e){
                        SKIP_BUILD = true;
                    }
                }
            }
        }
        stage ("Build"){
            agent {label 'XXX'}
            when {
                expression {SKIP_BUILD == false}
            }
.....

Took me some time to get all those strings together. Really hope you guys improve your CI plugins

Cheers, Kris
 

[manu]

18 hours ago, KrisK said:

Really hope you guys improve your CI plugins

It's about to happen, keep tuned!