Jump to content

Processing needed paths - taking very long time for large check-ins


bionicbeagle

Recommended Posts

   Hi,

 

   I'm evaluating PlasticSCM as an alternative to Perforce. I'm currently using the 4.2 prerelease (399), using SQL Server 2012 Enterprise for back-end storage.

 

   We run pretty large projects, so our scale probably differs considerably from a more typical user. Total repo size for this particular test is around 75GB, with close to 200k files. Our target use case is actually larger though (AAA commercial game development).

 

    One thing I noticed pretty quickly was that the initial submit was very slow to go through, due to what appears to be mostly client-side processing. For large submits (100k+ files) the client spends a lot of time at the "Processing needed paths" stage. For medium size submits (10k files) it goes through pretty quickly however. It's CPU bound on the client but I can't really see what it's doing. Looking at the .NET CLR performance counters the only thing I can see changing is the "# GC Handles" which goes up steadily. It's not doing any I/O or significant memory allocations by the looks of it. Once the client gets unstuck the submit process proceeds as expected... it's pretty odd.

 

   The critical path thread which is spinning has this call stack most of the time, which I can't make much sense of (via Process Hacker 2):

 

 

0, ntoskrnl.exe!SeAccessCheck+0x1ef
1, ntoskrnl.exe!KeDelayExecutionThread+0xc32
2, ntoskrnl.exe!KeWaitForSingleObject+0x1cf
3, ntoskrnl.exe!_misaligned_access+0x809
4, ntoskrnl.exe!SeAccessCheck+0x280
5, ntoskrnl.exe!SeAccessCheck+0x4f1
6, mscorwks.dll!IEE+0x392d
7, mscorwks.dll!PreBindAssembly+0x335f2
8, System.Globalization.CompareInfo.Compare(System.String, System.String, System.Globalization.CompareOptions) + 0x9a <-- mscorlib.ni.dll+0x36b7ca
9, rv.c(System.String, System.String) + 0x20 <-- 0x7fe8c557b30
10, rv.a(System.Collections.Generic.IList`1<System.String>, System.String) + 0x89 <-- 0x7fe8c9bc2c9
11, Codice.CM.Client.Gui.GuiChange.b(System.Collections.Generic.IList`1<Codice.CM.Client.Gui.GuiChange>) + 0xfc <-- 0x7fe8c9bad3c
12, mscorwks.dll!IEE+0xdde6
13, mscorwks.dll!CreateAssemblyNameObject+0x86183
14, mscorwks.dll!PreBindAssembly+0x7e5cf
15, mscorwks.dll!GetCLRFunction+0x9b42
16, mscorwks.dll!CorLaunchApplication+0x19d71
17, System.Reflection.RuntimeMethodInfo.Invoke(System.Object, System.Reflection.BindingFlags, System.Reflection.Binder, System.Object[], System.Globalization.CultureInfo, Boolean) + 0x4e7 <-- mscorlib.ni.dll+0x36b3e7
18, System.Delegate.DynamicInvokeImpl(System.Object[]) + 0x73 <-- mscorlib.ni.dll+0x3135a3
19, n4.i() + 0x5b <-- 0x7fe8c9ba46b
20, System.Threading.ExecutionContext.runTryCode(System.Object) + 0x178 <-- mscorlib.ni.dll+0x30dc78
21, mscorwks.dll!IEE+0xdde6
22, mscorwks.dll!CreateAssemblyNameObject+0x86183
23, mscorwks.dll!PreBindAssembly+0x7e5cf
24, mscorwks.dll!CreateAssemblyNameObject+0x7f312
25, mscorwks.dll!CorLaunchApplication+0x26802
26, System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object) + 0x62 <-- mscorlib.ni.dll+0x2f2ba2
27, System.Threading.ThreadHelper.ThreadStart() + 0x4d <-- mscorlib.ni.dll+0x38a9bd
28, mscorwks.dll!IEE+0xdde6
29, mscorwks.dll!CreateAssemblyNameObject+0x86183
30, mscorwks.dll!PreBindAssembly+0x7e5cf
31, mscorwks.dll!MetaDataGetDispenser+0xaadd
32, mscorwks.dll!InitializeFusion+0x948
33, mscorwks.dll!StrongNameFreeBuffer+0xeda5
34, mscorwks.dll!GetCLRFunction+0x845d
35, mscorwks.dll!CopyPDBs+0x20fa1
36, mscorwks.dll!InitializeFusion+0xae77
37, mscorwks.dll!InitializeFusion+0x2f48
38, kernel32.dll!BaseThreadInitThunk+0x1a
39, ntdll.dll!RtlUserThreadStart+0x21
 
Link to comment
Share on other sites

Hi bionicbeagle,

 

thanks for all the information! I agree with your thoughts, the GUI is doing more stuff and it can impact on the performance.

 

I do recommend you to use the CLI for this purposes, even more, you can use the "--timer" parameter to measure how long does the execution takes and send to us time time, number of objects and size.

 

I'll run a couple of test to review if the GUI is wasting time for unneeded things.

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...