How do you install use Semantic Merge plugins?

[Nymaen]

I found this (2 year old) plugin for F# and Semantic Merge. I would like to try and use it, and sense it is almost certainly out of date, try to get it working with current SemanticMerge. This implied there is some for of SDK I need too. Can anyone tell me what I need to get setup so I can make progress? A quick web search did not reveal anything about using plugins for SemanticMerge or anything about an SDK. 

 

Thanks in advance for the help!

[Gerard Murphy]

Hi Nymaen - I'm the person who wrote that plugin - I'm glad you're taking an interest.

 

You are right, I'm not actively developing that plugin any more, but please do fork the Git repo and take it further - that would be great. I'm happy to try to answer questions you may have in getting up to speed, so please post to this thread if you have more.

 

Semantic Merge (and Plastic SCM) use a different configuration these days to incorporate external parsers - there was a post on the Codice blog not so long ago - see http://codicesoftware.blogspot.com/2015/09/custom-languages-in-semantic-version.html.

 

That post is quite comprehensive - I use it myself for the companion Scala plugin, so on my machine I have the 'externalparsers.conf' file with contents of:-

 

.sc=C:\Workspaces\Neptunium\target\neptunium.cmd

.scala=C:\Workspaces\Neptunium\target\neptunium.cmd

 

In this example, 'neptunium.cmd' is a self-executing JAR with a Windows command script preamble that runs the plugin on Scala files of suffix '.scala' and '.sc'.

 

 

OK, that's for the Scala plugin - but you want to work on the F# plugin.

 

In your case, you would build the Visual Studio solution from your cloned Git repository, the solution contains two projects:-

 

1. FSharpPlugin - this is the driver program that contains a 'Main' that is run from either Semantic Merge or Plastic SCM in the manner described by that blog post I referenced above. Its job is to run in a loop and process requests from either product, handing off a pair of input and output file paths to the 'real' parser that is provided by the library 'StructureDiscovery'. The input file path will reference some file (might be a temporary one, or might be part of your workspace - it depends on what you are doing in the GUI, the point is that it should be an F# source file), the output file path will be the path that Plastic SCM / Semantic Merge expects the plugin to write the transcribed YAML for that source file to. This is a very simple piece of code that is really a shim around the real parser.

 

2. StructureDiscovery - this is a library where the real work takes place. Right now, there is a single module, 'FileProcessor' in it, that contains an entry point 'DiscoverStructure' - this is what the driver calls. 'DiscoverStructure' uses the open source F# compiler services framework to build an abstract syntax tree describing the input F# source file. See here http://fsharp.github.io/FSharp.Compiler.Service for an introduction. It then transforms that syntax tree into YAML by recursive decomposition, starting with the function 'yamlForOverallStructure' that is applied to the top of the syntax tree. There is a bit of fixing up of the syntax tree that is done before that function is called - this is done with 'adjustSpansToCoverInputFile', this makes sure that the spans cover *all* of the text in the file without leaving any holes - you get these holes when you use an abstract syntax tree, it's the same for both the F# and Scala parsers.

 

If you wondering what 'spans' are, read both the Codice documentation and the F# compiler services documentation - both use the same concept, although they have their own ways of representing them.

 

Anyway, you build the Visual Studio solution, and the resulting command line executable, 'FSharpPlugin.exe' is what you'd refer to in the external parsers config file, so you'd end up with something like this:-

 

 

.fs=C:\Workspaces\SemanticMergeFSharpPlugin\FSharpPlugin\bin\debug\FSharpPlugin.exe

.fsx=C:\Workspaces\SemanticMergeFSharpPlugin\FSharpPlugin\bin\debug\FSharpPlugin.exe

 

At this point, if you run Semantic Merge, it should pick up the plugin and do its stuff.

 

I'm guessing that the SDK you refer to is either the description of the plugin protocol - see the first link above for that, or the F# compiler services - see the second. The Visual Studio project uses NuGet to pull down the F# compiler services dependencies, so that should work straight out of the box.

 

If you run into problems with the configuration, the Plastic folks will definitely be able to help you, but I can also try to give you pointers.

 

 

All the best,

 

Gerard

[Gerard Murphy]

Three more things to mention:-

 

1. The F# Compiler Services library wasn't that well developed when I used this, but I'm fairly certain it's improved hugely since then. I would strongly recommend updating the NuGet dependency to track the latest version. There are other projects that use this library too, it's worth taking a look around to see what they do with it.

 

2. There is a comment in 'FileProcessor.fs':-

 

 

type LineSpan = pos * pos       // the second is a zero-relative character position within the line.   // The second sub-pair points one past the end of the construct,   // so the interval is [). It may either point past the end on the   // same line, or may point to the zeroeth character position on the   // next line, this depends on whether the construct is contained   // within one line or spans several.   type CharacterSpan = int * int // [] interval using zero-relative character position.  

The first part of this is *wrong* - the line spans used in the Semantic Merge specification are [] intervals - so the second sub-pair points to the end of the construct (so if the linespan is empty, it points one character in front of the start).

 

I fixed this bug in the Scala plugin, but never got round to this in the F# plugin. It might be a useful familiarisation exercise if you fixed this bug yourself - just bear in mind that the spans that come from the F# compiler service parser really do obey that comment - in other words, the spans work slightly differently in the F# compiler services and Semantic Merge.

 

If you read Scala, take a look at the Scala plugin, which is more developed port of the F# plugin - again, the Scala presentation compiler also has the same slightly different notion of spans, so the Scala plugin has to translate the spans - the F# plugin needs to have this fix back-ported into it.

 

3. What's definitely missing from the F# plugin is just a lot, a *lot*, more pattern matching on the various constructs of the abstract syntax tree - I stopped at the highest level, namely 'SynModuleDecl' for entire module constructs - what needs to be done is to decompose the sub-parts of the tree more to pull out classes and functions. Again, the Scala plugin does this and can be used as a reference, although I will say now that it's a lot easier to do this with the Scala presentation compiler than it was with the F# compiler services - but hopefully this may have changed on the F# side since I took a look. If not, it's a tedious but straightforward slog through lots of recursive pattern matching on the abstract syntax tree. I'm fairly sure that the F# source code formatter does exactly the same thing - I'd recommend taking a look at it, you may be able to either lift source code or even call it to do heavy lifting for you.

[Nymaen]

Thank you for the replies. I will take a look at this.