Jump to content
dummzeuch

Odd calls of external Pas2Yaml parser

Recommended Posts

I have got an odd issue with the the Pas2Yaml parser (modified to accept the additional encoding line):

It seems never to get the 'end' line:

This is the debug output:

READY
Received line: PathToTemp\u_RRMTP1b.pas-revBASE.svn00d.tmp.pas
Received encoding: Windows-1252
Received outputfile: PathToTemp\c525fb84-6c8e-4ec9-8e5f-98c290e1e165.tree
Parsed and writing to outputfile: PathToTemp\c525fb84-6c8e-4ec9-8e5f-98c290e1e165.tree
wrote OK
Received line: PathTo\src\u_RRMTP1b.pas
Received encoding: Windows-1252
Received outputfile: PathToTemp\b662819d-08d3-4be6-a680-db2067828443.tree
Parsed and writing to outputfile: PathToTemp\b662819d-08d3-4be6-a680-db2067828443.tree
wrote OK
Received line:
Received line:
Received line:

[and lots of more empty lines]

And that's it. After these lines it only gets empty lines and never gets the 'end' line.

I configured SemanticMerge for use as an external diff viewer in Tortoise svn like this:

PathTo\semanticmergetool.exe -s=%base -d=%mine -ep=PathTo\pas2yaml.exe

Based on this I would expect exactly two file sets being passed to pas2yaml:

1. The base file + its encoding + output file for it:

PathToTemp\u_RRMTP1b.pas-revBASE.svn00b.tmp.pas

Windows-1252

PathToTemp\a0f73627-d2e0-4662-9653-aac4a138994a.tree

2. My modified file + its encoding + output file for it:

PathTo\src\u_RRMTP1b.pas
Windows-1252
PathToTemp\57f72391-a859-45d2-bb17-cace52d03314.tree

Followed by a line containing 'end'.

Instead of the 'end' line I get an empty line for each call to ReadLn.

My question is: Are we sure that SemanticMerge 2.0 calls external parsers the way it is documented?

It is possible, that the created yaml files are not valid, I haven't checked that. Maybe SemanticMerge does not send the 'end' line in that case?

Share this post


Link to post
Share on other sites

Hi dummzeuch!

* I think it's better to try the parser step by step in order to determine the origin of the problem. 
That is, first only the parser then the integration with SemanticMerge and finally with Tortoise svn.

To check only the parser:

yourparserapplication.exe shell path_to_the_flag_file

More info: https://users.semanticmerge.com/documentation/external-parsers/external-parsers-guide.shtml#Asampleparsingsession

To check the integration with SemanticMerge:

semanticmergetool.exe -s sample\source.code -d sample\destination.code -ep=codeparser.exe

More info: https://users.semanticmerge.com/documentation/external-parsers/external-parsers-guide.shtml#SemanticMergeandSemanticDiff

* Sometimes, issues will happen if you forget to fill in the declarations tree correctly. If that happens, besides checking the error messages, looking at the logs might be very useful.
In this section, you can find the steps to activate the "ExternalParser" logging: https://users.semanticmerge.com/documentation/external-parsers/external-parsers-guide.shtml#Howtodebugyourparser

Tip: Maybe you can start with simplified source code, check it and, once it works, add the rest of the code.

In case the issue is not solved, you can share with us the parser in order to debug the SemanticMerge application to figure out what is happens.

Please do not hesitate to contact us if you have any questions.

Hope it helps!

Best regards,
Míryam 

Share this post


Link to post
Share on other sites

Hi,

thanks for your answer.

I have already tried the parser stand alone, which works fine as far as I can see.

For whatever reason I did not think of calling SemanticMerge directly. Must have been very tired...

Also, I will check the logging.

Share this post


Link to post
Share on other sites

I rewrote the logic of the main loop and now I get a diff display. But I still do not get the 'end' line. My program terminates after receiving 10 empty lines.

How do I configure logging and where are the log(s) placed?

The link says how to set the log level by creating a semantic.log.conf file, but not where this file has to be placed nor where the log file will be created.

I created the semantic.log.conf file in c:\users\<me>\AppData\Local\plastic4 with the suggested content (see attached file), but could not find a log file in %temp%. So where does it go?

 

 

semantic.log.conf

Share this post


Link to post
Share on other sites

OK, found it. There was already a semanticmerge.log.conf file in the SemanticMerge installation directory and this is also where the log file exists. (You should really add that to your instructions.)

Share this post


Link to post
Share on other sites

The log file does not help me. It does not contain any errors that I could identify. I have attached it to this message.

This is a single call to compare two files (but you'll probably see that yourself in the log).

The log file of pas2yaml reads like this:

READY
Received line: C:\Users\twm\AppData\Local\Temp\TestFile.pas-revBASE.svn001.tmp.pas
Received line: Windows-1252
Received line: C:\Users\twm\AppData\Local\Temp\b384da6b-a5cf-4cf7-8b1c-395446a37c98.tree
Received inputfile: C:\Users\twm\AppData\Local\Temp\TestFile.pas-revBASE.svn001.tmp.pas
Received encoding: Windows-1252
Received outputfile: C:\Users\twm\AppData\Local\Temp\b384da6b-a5cf-4cf7-8b1c-395446a37c98.tree
Parsed and writing to outputfile: C:\Users\twm\AppData\Local\Temp\b384da6b-a5cf-4cf7-8b1c-395446a37c98.tree
wrote OK
READY
Received line: D:\Source\_github\SemanticMergeDelphi\src\TestFile.pas
Received line: Windows-1252
Received line: C:\Users\twm\AppData\Local\Temp\b0fef9d2-98b2-4295-8f9c-7e7c2d7f8b6b.tree
Received inputfile: D:\Source\_github\SemanticMergeDelphi\src\TestFile.pas
Received encoding: Windows-1252
Received outputfile: C:\Users\twm\AppData\Local\Temp\b0fef9d2-98b2-4295-8f9c-7e7c2d7f8b6b.tree
Parsed and writing to outputfile: C:\Users\twm\AppData\Local\Temp\b0fef9d2-98b2-4295-8f9c-7e7c2d7f8b6b.tree
wrote OK
READY
Exception: Exception Exiting because INPUT reached EOF.

As you can see, the INPUT (=stdin) file gets closed before an 'end' line was received. This lets me suspect a bug in the way SemanticMerge calls the external parser: It probably closes the pipe to the external parser's stdin without first writing the 'end' line (or maybe simply forgets the CR/LF after it?).

semantic.log.txt.zip

Share this post


Link to post
Share on other sites

I reviewed the source code of the SemanticMerge application and the pas2yaml parser.

The 'end' string is only sent when SemanticMerge is launched through our version control tools (Plastic SCM and gmaster).

If I am correct, I think the "Exception Exiting because INPUT reached EOF" exception occurs when you close SemanticMerge due the process is not closed correctly using the 'end' string.

We will fix it as soon as possible. Meanwhile, you probably could workaround the issue avoid throwing the exception.

Does it sound good for you?

We will keep you informed as soon as it is ready!

Thanks for your feedback.

Best regards,
Míryam

Share this post


Link to post
Share on other sites
38 minutes ago, miryamgsm said:

The 'end' string is only sent when SemanticMerge is launched through our version control tools (Plastic SCM and gmaster).

If I am correct, I think the "Exception Exiting because INPUT reached EOF" exception occurs when you close SemanticMerge due the process is not closed correctly using the 'end' string.

Yes, that was my impression as well.

38 minutes ago, miryamgsm said:

We will fix it as soon as possible. Meanwhile, you probably could workaround the issue avoid throwing the exception.

Of course I can do that. My main problem with this was that the parser never exited because it kept reading empty lines. That stdin might have already been closed occurred to me only later.

May I suggest some changes to the API (for the next version of it):

1. In addtion to the shell parameter pass the API version so an external parser can check if it supports it, e.g.:

parser.exe shell 2 flagfile

The pas2yaml parser apparently originally did not support the encoding line but instead interpreted it as the output file which could result in it overwriting uncommitted changes (see my issue on the github page).

2. Also, I think it would be worthwile to not only pass the file names but prefix them with their meaning which could also prevent this kind of problem, e.g.:

InputFile: <input file goes here>

Encoding: <encoding goes here>

OutputFile: <output file goes here>

This won't add much overhead but make the communication much more reliable.

3. And last but not least: Require that an external parser does not to overwrite an existing output file. I think that shouldn't create any new issues since SemanticMerge always creates uniqe file names for each call, so these files should never exist in the first place. But again, this would prevent the issue of unwittingly overwriting files because the input was misinterpreted.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×