dummzeuch Posted January 2, 2019 Report Share Posted January 2, 2019 I have got an odd issue with the the Pas2Yaml parser (modified to accept the additional encoding line): It seems never to get the 'end' line: This is the debug output: READY Received line: PathToTemp\u_RRMTP1b.pas-revBASE.svn00d.tmp.pas Received encoding: Windows-1252 Received outputfile: PathToTemp\c525fb84-6c8e-4ec9-8e5f-98c290e1e165.tree Parsed and writing to outputfile: PathToTemp\c525fb84-6c8e-4ec9-8e5f-98c290e1e165.tree wrote OK Received line: PathTo\src\u_RRMTP1b.pas Received encoding: Windows-1252 Received outputfile: PathToTemp\b662819d-08d3-4be6-a680-db2067828443.tree Parsed and writing to outputfile: PathToTemp\b662819d-08d3-4be6-a680-db2067828443.tree wrote OK Received line: Received line: Received line: [and lots of more empty lines] And that's it. After these lines it only gets empty lines and never gets the 'end' line. I configured SemanticMerge for use as an external diff viewer in Tortoise svn like this: PathTo\semanticmergetool.exe -s=%base -d=%mine -ep=PathTo\pas2yaml.exe Based on this I would expect exactly two file sets being passed to pas2yaml: 1. The base file + its encoding + output file for it: PathToTemp\u_RRMTP1b.pas-revBASE.svn00b.tmp.pas Windows-1252 PathToTemp\a0f73627-d2e0-4662-9653-aac4a138994a.tree 2. My modified file + its encoding + output file for it: PathTo\src\u_RRMTP1b.pas Windows-1252 PathToTemp\57f72391-a859-45d2-bb17-cace52d03314.tree Followed by a line containing 'end'. Instead of the 'end' line I get an empty line for each call to ReadLn. My question is: Are we sure that SemanticMerge 2.0 calls external parsers the way it is documented? It is possible, that the created yaml files are not valid, I haven't checked that. Maybe SemanticMerge does not send the 'end' line in that case? Link to comment Share on other sites More sharing options...
dummzeuch Posted January 2, 2019 Author Report Share Posted January 2, 2019 The created yaml seems to be correct as far as I can determine. I'm out of ideas now. Link to comment Share on other sites More sharing options...
miryamgsm Posted January 4, 2019 Report Share Posted January 4, 2019 Hi dummzeuch! * I think it's better to try the parser step by step in order to determine the origin of the problem. That is, first only the parser then the integration with SemanticMerge and finally with Tortoise svn. To check only the parser: yourparserapplication.exe shell path_to_the_flag_file More info: https://users.semanticmerge.com/documentation/external-parsers/external-parsers-guide.shtml#Asampleparsingsession To check the integration with SemanticMerge: semanticmergetool.exe -s sample\source.code -d sample\destination.code -ep=codeparser.exe More info: https://users.semanticmerge.com/documentation/external-parsers/external-parsers-guide.shtml#SemanticMergeandSemanticDiff * Sometimes, issues will happen if you forget to fill in the declarations tree correctly. If that happens, besides checking the error messages, looking at the logs might be very useful. In this section, you can find the steps to activate the "ExternalParser" logging: https://users.semanticmerge.com/documentation/external-parsers/external-parsers-guide.shtml#Howtodebugyourparser Tip: Maybe you can start with simplified source code, check it and, once it works, add the rest of the code. In case the issue is not solved, you can share with us the parser in order to debug the SemanticMerge application to figure out what is happens. Please do not hesitate to contact us if you have any questions. Hope it helps! Best regards, Míryam Link to comment Share on other sites More sharing options...
dummzeuch Posted January 4, 2019 Author Report Share Posted January 4, 2019 Hi, thanks for your answer. I have already tried the parser stand alone, which works fine as far as I can see. For whatever reason I did not think of calling SemanticMerge directly. Must have been very tired... Also, I will check the logging. Link to comment Share on other sites More sharing options...
dummzeuch Posted January 4, 2019 Author Report Share Posted January 4, 2019 I rewrote the logic of the main loop and now I get a diff display. But I still do not get the 'end' line. My program terminates after receiving 10 empty lines. How do I configure logging and where are the log(s) placed? The link says how to set the log level by creating a semantic.log.conf file, but not where this file has to be placed nor where the log file will be created. I created the semantic.log.conf file in c:\users\<me>\AppData\Local\plastic4 with the suggested content (see attached file), but could not find a log file in %temp%. So where does it go? semantic.log.conf Link to comment Share on other sites More sharing options...
dummzeuch Posted January 4, 2019 Author Report Share Posted January 4, 2019 OK, found it. There was already a semanticmerge.log.conf file in the SemanticMerge installation directory and this is also where the log file exists. (You should really add that to your instructions.) Link to comment Share on other sites More sharing options...
dummzeuch Posted January 4, 2019 Author Report Share Posted January 4, 2019 The log file does not help me. It does not contain any errors that I could identify. I have attached it to this message. This is a single call to compare two files (but you'll probably see that yourself in the log). The log file of pas2yaml reads like this: READY Received line: C:\Users\twm\AppData\Local\Temp\TestFile.pas-revBASE.svn001.tmp.pas Received line: Windows-1252 Received line: C:\Users\twm\AppData\Local\Temp\b384da6b-a5cf-4cf7-8b1c-395446a37c98.tree Received inputfile: C:\Users\twm\AppData\Local\Temp\TestFile.pas-revBASE.svn001.tmp.pas Received encoding: Windows-1252 Received outputfile: C:\Users\twm\AppData\Local\Temp\b384da6b-a5cf-4cf7-8b1c-395446a37c98.tree Parsed and writing to outputfile: C:\Users\twm\AppData\Local\Temp\b384da6b-a5cf-4cf7-8b1c-395446a37c98.tree wrote OK READY Received line: D:\Source\_github\SemanticMergeDelphi\src\TestFile.pas Received line: Windows-1252 Received line: C:\Users\twm\AppData\Local\Temp\b0fef9d2-98b2-4295-8f9c-7e7c2d7f8b6b.tree Received inputfile: D:\Source\_github\SemanticMergeDelphi\src\TestFile.pas Received encoding: Windows-1252 Received outputfile: C:\Users\twm\AppData\Local\Temp\b0fef9d2-98b2-4295-8f9c-7e7c2d7f8b6b.tree Parsed and writing to outputfile: C:\Users\twm\AppData\Local\Temp\b0fef9d2-98b2-4295-8f9c-7e7c2d7f8b6b.tree wrote OK READY Exception: Exception Exiting because INPUT reached EOF. As you can see, the INPUT (=stdin) file gets closed before an 'end' line was received. This lets me suspect a bug in the way SemanticMerge calls the external parser: It probably closes the pipe to the external parser's stdin without first writing the 'end' line (or maybe simply forgets the CR/LF after it?). semantic.log.txt.zip Link to comment Share on other sites More sharing options...
dummzeuch Posted January 4, 2019 Author Report Share Posted January 4, 2019 Just in case anybody is interested, here is my current source code: https://github.com/dummzeuch/pas2yaml Link to comment Share on other sites More sharing options...
miryamgsm Posted January 4, 2019 Report Share Posted January 4, 2019 I reviewed the source code of the SemanticMerge application and the pas2yaml parser. The 'end' string is only sent when SemanticMerge is launched through our version control tools (Plastic SCM and gmaster). If I am correct, I think the "Exception Exiting because INPUT reached EOF" exception occurs when you close SemanticMerge due the process is not closed correctly using the 'end' string. We will fix it as soon as possible. Meanwhile, you probably could workaround the issue avoid throwing the exception. Does it sound good for you? We will keep you informed as soon as it is ready! Thanks for your feedback. Best regards, Míryam Link to comment Share on other sites More sharing options...
dummzeuch Posted January 4, 2019 Author Report Share Posted January 4, 2019 38 minutes ago, miryamgsm said: The 'end' string is only sent when SemanticMerge is launched through our version control tools (Plastic SCM and gmaster). If I am correct, I think the "Exception Exiting because INPUT reached EOF" exception occurs when you close SemanticMerge due the process is not closed correctly using the 'end' string. Yes, that was my impression as well. 38 minutes ago, miryamgsm said: We will fix it as soon as possible. Meanwhile, you probably could workaround the issue avoid throwing the exception. Of course I can do that. My main problem with this was that the parser never exited because it kept reading empty lines. That stdin might have already been closed occurred to me only later. May I suggest some changes to the API (for the next version of it): 1. In addtion to the shell parameter pass the API version so an external parser can check if it supports it, e.g.: parser.exe shell 2 flagfile The pas2yaml parser apparently originally did not support the encoding line but instead interpreted it as the output file which could result in it overwriting uncommitted changes (see my issue on the github page). 2. Also, I think it would be worthwile to not only pass the file names but prefix them with their meaning which could also prevent this kind of problem, e.g.: InputFile: <input file goes here> Encoding: <encoding goes here> OutputFile: <output file goes here> This won't add much overhead but make the communication much more reliable. 3. And last but not least: Require that an external parser does not to overwrite an existing output file. I think that shouldn't create any new issues since SemanticMerge always creates uniqe file names for each call, so these files should never exist in the first place. But again, this would prevent the issue of unwittingly overwriting files because the input was misinterpreted. Link to comment Share on other sites More sharing options...
miryamgsm Posted January 9, 2019 Report Share Posted January 9, 2019 We will take into account your suggestions for future improvements. Thanks for your feedback. Best regards, Míryam Link to comment Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.