Object Comparison - faster options? Using it wrong?

Feb 14, 2011 at 4:53 PM
Edited Feb 14, 2011 at 4:55 PM

Hi,

I've just started using the object comparison part of TestApi, and I've noticed that it's quite slow, at least for me. An object comparer that I wrote myself that also uses reflection is doing a set of tests (comparing various complex bespoke objects and exceptions) in around 3-4 seconds, wheras TestApi's functions take around 20 seconds to run the same tests. Mine does give up a little earlier when presented with indexed properties that either have ambiguous length fields or don't have one of the default names for a length field though.

Is using IEnumerable for indexes and that sort of thing really where all that slowdown is as I suspect, or am I doing something wrong? Here's the code that actually uses TestApi.

using Microsoft.Test.ObjectComparison;
private static IList<string> CompareObjectProperties(object left, object right)
        {
            ObjectGraphFactory fac = new PublicPropertyObjectGraphFactory();
            ObjectGraphComparer comparer = new ObjectGraphComparer();
            IEnumerable<ObjectComparisonMismatch> mismatches;
            IList<string> differences = new List<string>();

            GraphNode leftNode = fac.CreateObjectGraph(left);
            GraphNode rightNode = fac.CreateObjectGraph(right);

            bool noDifferences = comparer.Compare(leftNode, rightNode, out mismatches);

            if (noDifferences)
            {
                return new List<string>(); // no differences to mention, so empty list.
            }
            else
            {
                foreach (ObjectComparisonMismatch mismatch in mismatches)
                {
                    // format stuff differently depending on what type of mismatch it is.
                    switch (mismatch.MismatchType)
                    {
                        case ObjectComparisonMismatchType.ObjectTypesDoNotMatch:
                            differences.Add(mismatch.LeftObjectNode.QualifiedName + ": Types do not match. Left = " + mismatch.LeftObjectNode.ObjectType.ToString() + "\tRight = " + mismatch.RightObjectNode.ObjectType.ToString());
                            break;

                        case ObjectComparisonMismatchType.ObjectValuesDoNotMatch:
                            differences.Add(mismatch.LeftObjectNode.QualifiedName + ": Values do not match. Left = " + mismatch.LeftObjectNode.ObjectValue.ToString() + "\tRight = " + mismatch.RightObjectNode.ObjectValue.ToString());
                            break;

                        case ObjectComparisonMismatchType.MissingLeftNode:
                            differences.Add(mismatch.RightObjectNode.QualifiedName + ": missing in left node");
                            break;

                        case ObjectComparisonMismatchType.MissingRightNode:
                            differences.Add(mismatch.LeftObjectNode.QualifiedName + ": missing in right node");
                            break;

                        default:
                            differences.Add(mismatch.LeftObjectNode.QualifiedName + ": " + mismatch.MismatchType.ToString());
                            break;
                    }
                }

                return differences;
            }
        }

If all is indeed working as intended, then I think it would be a good idea to include a faster version that as a tradeoff for increased speed doesn't go into as much detail, along with the current set of defaults. I assume it would be another type of pre-written ObjectGraphFactory.

private static IList<string> CompareObjectProperties(object left, object right)
        {
            ObjectGraphFactory fac = new PublicPropertyObjectGraphFactory();
            ObjectGraphComparer comparer = new ObjectGraphComparer();
            IEnumerable<ObjectComparisonMismatch> mismatches;
            IList<string> differences = new List<string>();

            GraphNode leftNode = fac.CreateObjectGraph(left);
            GraphNode rightNode = fac.CreateObjectGraph(right);

            bool noDifferences = comparer.Compare(leftNode, rightNode, out mismatches);

            if (noDifferences)
            {
                return new List<string>(); // no differences to mention, so empty list.
            }
            else
            {
                foreach (ObjectComparisonMismatch mismatch in mismatches)
                {
                    // format stuff differently depending on what type of mismatch it is.
                    switch (mismatch.MismatchType)
                    {
                        case ObjectComparisonMismatchType.ObjectTypesDoNotMatch:
                            differences.Add(mismatch.LeftObjectNode.QualifiedName + ": Types do not match. Left = " + mismatch.LeftObjectNode.ObjectType.ToString() + "\tRight = " + mismatch.RightObjectNode.ObjectType.ToString());
                            break;

                        case ObjectComparisonMismatchType.ObjectValuesDoNotMatch:
                            differences.Add(mismatch.LeftObjectNode.QualifiedName + ": Values do not match. Left = " + mismatch.LeftObjectNode.ObjectValue.ToString() + "\tRight = " + mismatch.RightObjectNode.ObjectValue.ToString());
                            break;

                        case ObjectComparisonMismatchType.MissingLeftNode:
                            differences.Add(mismatch.RightObjectNode.QualifiedName + ": missing in left node");
                            break;

                        case ObjectComparisonMismatchType.MissingRightNode:
                            differences.Add(mismatch.LeftObjectNode.QualifiedName + ": missing in right node");
                            break;

                        default:
                            differences.Add(mismatch.LeftObjectNode.QualifiedName + ": " + mismatch.MismatchType.ToString());
                            break;
                    }
                }

                return differences;
            }
Developer
Feb 15, 2011 at 1:22 AM

Hi Ben,

Thank you for reporting the issue. I do not know what types of objects are compared, but the slowdown can certainly be due to the fact that PublicPropertyObjectGraphFactory recursively enumerates through all public properties of a passed object. I would recommend to use a stub factory to avoid creation of graph nodes for objects you are not interested in. That can be achieved by factory specializations introduced in the current release. For example,

    class Program
    {
        static void Main(string[] args)
        {
            var left = typeof(string);
            var right = typeof(string);

            ObjectGraphFactory fac = new PublicPropertyObjectGraphFactory();
            ObjectGraphComparer comparer = new ObjectGraphComparer();
            IEnumerable<ObjectComparisonMismatch> mismatches;
            IList<string> differences = new List<string>();

            var factoryMap = new ObjectGraphFactoryMap(false);
            factoryMap.Add(typeof(MethodBase), new StubGraphFactory());
            factoryMap.Add(typeof(Assembly), new StubGraphFactory());
            
            GraphNode leftNode = fac.CreateObjectGraph(left, factoryMap);
            GraphNode rightNode = fac.CreateObjectGraph(right, factoryMap);
            
            bool noDifferences = comparer.Compare(leftNode, rightNode, out mismatches);

            // By using StubFactory chindrenCount is reduced from 413 to 251.
            var chindrenCount = leftNode.GetNodesInDepthFirstOrder().Count();
        }
    }

    class StubGraphFactory : ObjectGraphFactory
    {
        public override GraphNode CreateObjectGraph(object value, ObjectGraphFactoryMap factoryMap = null)
        {
            return new GraphNode 
            { 
                Name = value == null ? "null" : value.GetType().Name, 
                ObjectValue = value 
            };
        }
    }

 

So, the number of graph nodes is reduced by 413 – 251 = 162. I assume that processing time will decrease proportionally. 

 There is a little complication though. There was a bug in PublicPropertyObjectGraphFactory which used to make the approach above not workable. The bug was fixed in the changeset 58071. Please sync to this changeset if you already have downloaded TestApi sources, otherwise you will need to do so. I am sorry for any inconvenience.

Thanks,
Andrey