Pex and Moles to be Visual Studio 11 Feature Set

In true, "I will neither confirm nor deny" Microsoft fashion, a little bird in Redmond informed me that Pex and Moles will have no further releases, because they are being "productized". Another, larger bird stated, "dependency isolation will be taken care of, in Visual Studio 2012," when I asked if Moles and Pex will be rolled into VS 2012.

The Visual Studio 2012 community technical preview (CTP) is available for download to anybody.  However, the Moles and Pex products are not yet integrated into the application.  There are several points of integration the need to be perfected, before adding it.

Note, this VS11 download is a BETA version, and is incomplete.  Many more features are still cooking.  Stability is OK, but use it at your own risk!  Visual Studio 2012 Beta will (should) receive updates from Microsoft Update, to keep it current with the newest release.


(This post is extracted from the Testing with Microsoft Pex and Moles page of this blog.  I felt this information was worth putting in its own post, after sitting in a sidebar, since May 2011.)

UPDATE: Changed the link and name to reflect the Beta release.

How To Mole the System Assembly

The System namespace may be moled, in addition to any other child namespace.  For instance, you may want to detour calls away from System.StringComparer, which is a member of the System assembly.   To mole members of the .NET Framework "System" namespace, follow these four steps:

How To Detour a Static Constructor

Static constructors pose a challenge.  Naturally, one can not call the constructor of a static class.  This also means that the Moles Framework does not provide an accessor to detour the constructor of a static class.  However, Moles provides a way to erase static constructors, to prevent them from executing.  Simply add the MolesDisableStaticConstructor assembly attribute to the test class:

[assembly: MolesEraseStaticConstructor(typeof(MyStatic))]

Partial Stubs

Partial stubs are used when you want to stub only some of the members of a stub or mole type, and allow the others to fall through to their original code.  This is useful, when only some of the members of a class handle a dependency object, but not others.  After all, the purpose of Moles is to isolate dependencies.  It is always best to leave the smallest footprint possible, when dealing with isolation.

(See

How Do I Detour a Mole Type Constructor?

Please be aware this information is based on Moles version v0.94.51023.0, 10/23/2010, and may not be accurate for prior versions.

Stub type constructors are not able to be detoured; because, they should be instantiated in the test, and have no need to be detoured.  Subsequently, no constructor detour appears in the stub type.

Constructors are located in the mole type root, and are named "Constructor", followed by the argument type names.  For example, the following are constructor detours for System.IO.Moles.MFile:

How Do I Write a Detour?

If you don't use lambda expressions, take a couple of hours right now to learn it, and vastly improve your skill set.  I can wait...

Now that we all know lambda expressions are simple delegates set to "super easy" mode, you already know how to write a detour!  Detours are set the same way in both stub and mole types.  You have two options, when detouring a call to a stub or moles type member.

Granted, these code examples are very simplistic, but you can use these to your advantage. See the Tricks With Detours section of the Testing With Microsoft Pex and Moles page, for some ideas.

Why Are Moles Assemblies Still Added To My Project?

Most people using moles, like Dave, encounter the situation where they want to remove Moles from the project.  You delete the hidden MolesAssemblies folder, but Moles just won't go away!  How do you remove moles from a project?

When Do I Use a Stub or Mole Type?

Give preference to Stub types.  Using a stub type helps you adhere to sound architecture principles, when writing new code and/or using Test Driven Development (TDD).

What Are Mole and Stub Types?

Moles generates two types, in the Moles assemblies: Mole types and Stub types.  Both behave very much the same, but are are used in mutually exclusive situations.  See page 3 of the Moles Reference Manual, for more information on stub and mole types.

An important concept often overlooked by developers investigating the Moles Framework is that the mole and Stub types may be configured with frequently-used detours, in a private, parameterized test class method.  This allows you to centralize the setting of frequently-used values, and therefore adhering to the DRY principle.

How Do Moles Detours Work?

Moles is a dependency isolation and mocking framework.  Instead of building mock stubs for dependency injection, Moles allows the programmer to detour the call to portions of the actual dependency object.  So, how does Moles detour calls, without touching the source code?

What is a Call Detour?

Detouring a call means the call to the target object (method, property, etc.) is redirected to another object.  That other object will be an anonymous method or other method in the test class.  Detours are delegates, generally managed through a Func< , > object.

What are .MOLES Files?

After clicking the Add Moles Assembly option, in the Solution Explorer context menu, a file with extension .moles is added to the test project.  This file contains an instruction for the compiler, that requests a mole assembly to be generated for the specified assembly.  Without these files, no moles assemblies will be generated.

Moles Error Says I Haven't Instrumented a Type. What Does That Mean?

There are three possible causes for instrumentation exceptions:

Using Declarations and Assembly Attributes
You must ensure a MoledType assembly attribute is present, for the type specified by the exception.  For example:

[assembly: MoledType(typeof(System.IO.File))]

Refer to the How Do I Make My Test Use Moles post, for more details.


Mode of Execution
If you are certain you have included all of the required using declarations and assembly attributes, examine your method of execution.  Pex and Moles tests require special instructions to be sent to the compiler, to properly route calls through the Moles assembly.  Third party test execution utilities, such as CodeRush, do not provide this supplemental information.  A missing instrumentation exception will be thrown, when Pex and Moles tests are executed by CodeRush.


Unsupported Test Framework
Pex and Moles support both the Visual Studio and NUnit Test Frameworks.  Other frameworks are not supported.

Why Doesn't the .Moles Assembly Appear in Solution Explorer?

There are two common answers to this question:

1. Compile the test project.  The moles assemblies are generated on compile.
2. Select the project in the Solution Explorer window, and then click the Show All Files button.  A hidden folder named MolesAssemblies will appear.

There should be no reason to handle the moles assemblies, since they may be frequently regenerated.

Where Are the Moles Assemblies?

"I compiled my test project, after asking to mole an assembly.  I see the .moles file, but where is the moles assembly?"

Moles assemblies are visible in the Solution Explorer window, by clicking the Show All Files button.  (Be sure the proper project is selected, first!) All moles assemblies are found in the hidden MolesAssemblies folder.

How Do I Make My Build and/or Test Servers Use Pex and Moles?

All build and test servers (e.g. MSBuild) that must handle Pex and/or Moles have the Pex and Moles Framework installed.  After installation, the builds and tests should execute normally.

How do I make my test use Moles?

NOTE: The Moles isolation framework must be installed to any machine that will compile the solution and/or execute the mole tests.

The following things must be done, to use Moles in a test.  This looks like a lot, now.  However, you'll quickly get the hang of things, and have these steps done in seconds!  Using my CodeRush Moles templates will help speed things up.

How Do I Mole .NET Framework Types?

Moles ships with pre-compiled moles assemblies for the entire .NET Framework (mscorlib).  (This used to be a huge pain to compile!)  Adding the moled mscorlib is easy.  Just follow these three steps:

How do I Mole an Assembly?

Creating a Moles wrapper assembly ("moled" assembly) has been greatly simplified:

1. Expand the test project node, in the Solution Explorer window
2. Expand the References child node
3. Right-click the assembly you wish to mole -- the context menu appears
4. In the context menu, select the Add Moles Assembly option -- a .moles file appears in the text project
5. Build the test project

What is Pex and Moles?

UPDATE
Please be aware the Moles Framework has been productized and released as Microsoft Fakes Framework, in the Visual Studio 2012 Ultimate and Premium (with Update 2) SKUs. Pex seems to have evaporated -- I hear no news regarding Pex, from the internals of Microsoft. This information is intended for historical reference for developers that still use or encounter these technologies.


This is the first post in a series. Please visit the new Testing wih Pex and Moles page, for an index to all related posts.

Microsoft Pex and Moles are a set of automated white box unit testing utilities, developed by Microsoft Research in Software (RiSE) group.  As of writing this post, the Pex and Moles package is version 0.94.51023.0, and has been in development for several years.  The first public release of Pex was released in mid 2008.  Moles debuted as a host type, in late 2009.  These both matured greatly, until October 2010.  Since that time, no subsequent releases have been issued.  This could mean the technologies are being polished up for release, or waiting for an opportunity for release with another project like Visual Studio 11.  There are still some seemingly obvious features that are missing, so the case could also be that it is simply put on the back burner for a while.  However, the late addition of MSBuild support suggests otherwise.


UPDATE (21 OCT 2011):
I suspected that something was going on with Pex and Moles, since the releases are running out of beta numbers (0.92.xxx), and nothing new has been released for a year.  This typically indicates the project has been handed off to another team for integration in to a release product.

I asked a very respectable member of the Microsoft Visual Studio development team, if Pex/Moles or some other dependency isolation mechanism will be introduced in Visual Studio 11. In traditional Microsoft "I will nether confirm nor deny" fashion, this individual told me, "Dependency isolation will be taken care of, in Visual Studio 11."  It appears Pex and Moles are finally being rolled into Visual Studio!  The current VS11 preview does not yet include dependency isolation features.  Hopefully, this information I have blogged will become obsolete!

Note, this VS11 download is a PREVIEW, not a BETA, or even an ALPHA.  Many more features are still cooking.  Stability is OK, but use it at your own risk!

The Toolbar and Blank Key Challenges

I issue two challenges, in this post.  The first gives you geek points; the second geek cred; and both together, a substantial step forward in becoming an ubergeek.

1. Turn off all toolbars in your Visual Studio.
I've never used them (admittedly, I do have the Solution Configurations" and Stop Debugging commands in the menu bar), and didn't realize how often many developers use them.  I was working a Kata session at Utah Code Camp and SQL Saturday 94, this last weekend, with a fellow who used Cut and Paste commands in the context menu.  That may be a step or two beyond "crutch" status.  Incidentally, Scott Hanselman posted something similar to his blog, earlier this year.

Note, we both disabled the navigation bar at the top of documents (Tools > Options > Text Editor > All Languages > General > Navigation bar).

2. Get (and use) a keyboard with no key labels.
At the aforementioned  Utah Code Camp, David Adist showed up in the speaker green room with a blank keyboard.  Although I had seen them before, I never though I could use one.  He claims that experienced programmers already have a good sense of touch-type; but, you can't exercise your ability when you can cheat by looking at the keys.

Das Keyboard has two models: silent and mechanical.  The mechanical version gives you that great, "clacky" sound that makes you sound like you're typing very fast (even though half of those keystrokes are the backspace key!).

Pex & Moles CodeRush Templates Version 1.1

Download Mike's updated CodeRush templates, here:  http://bit.ly/nj1hMy

What's new in Mike's CodeRush Templates v. 1.1:

• New Templates
• aiot = Assert.IsInstanceOfType(object value, Type expectedType)
• ainot = Assert.IsNotInstanceOfType(object value, Type expectedType)
• Updates
• [mt - automatically inserts type of object on clipboard
• [hm, [ma, [mt, [pm - Now suppress insertion of an extra ] character at end of the line
• tm and tmm - Removed marker in the Arrange section
• tmm - Now inserts [HostType("Moles")] above [TestMethod()], for template consolidation reasons
• Updated field information box content, in several templates

Pex and Moles DevExpress CodeRush Templates

CodeRush Xpress (or with RefactorPro), from DevExpress, is probably the most useful and well-built IDE add-ons I have ever used. The templates are extremely useful, especially when you can modify them and create your own. Here are a few templates that I have created for Pex and Moles.

Click here, to download the templates.

NOTE: These templates are for the Microsoft Test Framework, and will not work when in an NUnit project.  The template shortcut is highlighted in yellow, followed by a description and an example of its output.

How to Clear DevExpress CodeRush Assembly and Solution Cache

My employer has been wrestling with a juggernaut-class ASP.NET application, complete with an intermediary framework, that is a port of the soon-to-be-retired mainframe.  This is a colossal, 4GB monster with procedural C# files, frequently exceeding 30,000 lines.  Needless to say, CodeRush gets pretty busy, when we get a new revision from the external conversion company, every 2 weeks.

With 25GB of cache data, CodeRush really starts to drag down performance; WAY more cache data than most developers dreamed of.  Blowing the cobwebs out of DevExpress CodeRush cache every month or two helps keep things humming along nicely, around the office.

There are a few ways to clear out the assembly cache and solution cache:

Assembly Not Available in the Currently Targeted Framework

Oh, dreaded .NET 4.0 target framework issue... how I loathe thee.

Symptoms

When attempting to compile a .NET application, using Visual Studio 2010, the build fails, and the following warnings, or similar warnings, are issued:

The primary reference "System.Configuration.Install", which is a framework assembly, could not be resolved in the currently targeted framework. ".NETFramework,Version=v4.0". To resolve this problem, either remove the reference "System.Configuration.Install" or retarget your application to a framework version which contains "System.Configuration.Install". 

The primary reference "System.ServiceProcess", which is a framework assembly, could not be resolved in the currently targeted framework. ".NETFramework,Version=v4.0". To resolve this problem, either remove the reference "System.ServiceProcess" or retarget your application to a framework version which contains "System.ServiceProcess".

I found that even though the .NET Client profiles are supposed to be a subset of the full profile, many assemblies, including the two noted above, are not included in the "Add Reference" dialog.  These assemblies properly appear in the Client Profile.  It's as if the list has somehow been modified.  Now, to identify the culprit...

Introducing Moles and PEX

*NOTE: I am compiling materials for Pex/Moles presentations, and will post them here.  Production of the remainder of this series of posts is on hold, until then.  Sorry!

---

Microsoft's Research in Software Engineering (RiSE) has been developing a pair of unit testing technologies for the past several years, named MOLES and PEX.  Awareness and pre-market adoption of these technologies is being increasingly adopted by the programming community.

UPDATE:  Please see the new Testing with Pex and Moles page, that contains links to all related posts!

How to Mole a Specific Type Instance

So, you're using Moles for unit test dependency isolation.  You have a method to test that (unavoidably) uses several instances of the same type.  You determine a unit test must be created to test that each instance must be tested individually.  Moles is perfect for detouring calls to the API or .NET Framework dependency, but how do you mole a single instance of the common type, instead of all instances of the type?

Moles requires tests to be IN an instrumented process

After adopting Moles as my dependency isolation utility of choice for .NET projects, I began to encounter an exception that could not be explained by anyone, including Peli:

Moles requires tests to be IN an instrumented process.

To compound my frustration, my unit tests involving "moled" types began to work sometimes, and would work other times.  To add frustration, no code changes took place between the successful and unsuccessful test executions.

But something had indeed changed.  Not in the code.  Not on my workstation. Not in Visual Studio 2010 or Team Foundation Server.  How I executed the text changed.

How to Stub Dependency Event Handlers in Integration Tests

Integration tests help you ensure dependencies behave as expected.  When creating unit tests, developers go to great lengths to eliminate dependencies, but that's not our objective.  The application should already have passed unit tests, before proceeding with integration tests. We want to ensure that integrating the application with its dependencies works as expected.

"Dependencies" includes other modules, APIs, frameworks, hardware, databases, networks, etc.  We want to ensure the application works with these other resources as expected.  This demonstration code implements constructor injection, to simplify testing.  Dependency injection is essential to unit testing, but also proves useful for integration testing.  The injection is important, because we want to isolate a specific dependency and inject a controlled mock object for others.

Unit Testing with Moles and Pex - Part 1

Tweet

This post discusses key ways of improving the effectiveness and code coverage on unit tests, using the Visual Studio Test Framework, including the developing Moles and Pex technologies.  Other frameworks and harnesses such as nUnit, Moq, etc. are not discussed in this post.

Moles is Microsoft's code instrumentation, or mock framework, used to isolate environment dependencies.  Moles was created expressly to enable Pex to function.  Moles is discussed in greater detail, after the break.

Pex intelligently examines the logic branches of the code to be tested, and then generates a series of input values that should or should not fail, thereby maximizing test code coverage.

Before we start talking about Moles and Pex, there are some elementary, yet vitally important topics to review about unit testing.  Adhering to these rules play into the effectiveness of Moles and Pex.  I strongly advise reading, before moving on to part 2, but I really can't stop you, can I? (or can I? Muahaha!)

ATOM and RSS Feeds Fixed

Yeah, my bad.  Again.

Apparently, the last amazing adjustment I made to the RSS and ATOM feed settings at FeedBurner.com didn't take well.  I have adjusted them yet again, and they appear to be functioning correctly, this time.  Sorry I was not aware of the issue, as I don't see a need to subscribe to my own blog.

Go ahead... try clicking the ATOM and.or RSS icons, in the sidebar.  They really, really work.  (For real!)

Active Directory Properties for C#

I compiled the following list of Active Directory properties, while developing a solution. This list is more comprehensive than those I found on the InterWebs.

Please submit suggestions for additions to this list, via the comment block, below!  Code thrives on community feedback. (Code is posted after the jump break...)

Dependency Management

In the world of programming, nothing guarantees an instant ulcer more than change. Object-oriented programming (OOP) is particularly susceptible to problems stemming from changes made to objects upon which the code is dependent. However, when dependencies are well managed, adverse affects from change can be kept at bay.

Fix TFS "Cloaked Path" Problem

Here's the situation you're likely facing: you've been tasked to add a Visual Studio (VS) solution to an Team Foundation Server (TFS) as a Team Project (TP). You launched Visual Studio or the TFS Web interface, and then navigated to the TP in the Source Control Explorer. You dragged the solution folder from Windows Explorer into Source Control Explorer. The Add to Source Control dialog appeared, but you notice the following:

• The Items to add tab says "(0)", indicating no items will be added
• All files are listed in the Exlcuded items tab
• The status message below the item tabs reads, "This item cannot be added because its destination path is cloaked"
• You are asking yourself, "What the heck is a cloaked path?"

You probably tried simply clicking the Finish button instead of the Next > button, but receive a message stating, "No eligible items were found for addition."

Well, there is an easy solution for that!

The Virtual PC Login Blockade

"Virtual PC Won't Let Me Log In!"

I've heard that one before.  So, you're using Microsoft Virtual PC for quality assurance (QA) testing, or for other use. Things are going fine, until after a reboot of the virtual machine (VM). When it boots, the VM operating system (OS) prompts you to log in. (See image.) No matter what you enter, the machine just won't log in! Even the local administrator credentials won't work.