Mark Gilbert's Blog

Science and technology, served light and fluffy.

My buffer runneth over

In the new release of NAntRunner, one of the new features is a checkbox to add the -verbose switch to the NAnt call. In the course of testing it, I found that NAnt would hang when I ran a non-trivial build and deploy script. When NAnt.exe hung, it locked up NAntRunner, too. If I ran that same script from the command line using NAnt (bypassing NAntRunner), it completed fine.

Through some tinkering I found that once NAnt.exe hung, I could kill it off using Task Manager and then NAntRunner would give me control back. I started doing some searching for “process.start hung” and “process.start doesn’t exit”, and eventually came to a couple of articles that talked about the output buffers filling up:

http://www.velocityreviews.com/forums/t123227-console-application-hangs-when-called-from-processstart.html

http://stackoverflow.com/questions/439617/hanging-process-when-run-with-net-process-start-whats-wrong

Both of these discuss redirecting the buffers for a spawned process. As it turns out, the buffers have a 2K limit and if they fill up, the process hangs. That sounded awfully familiar.

All versions of NAntRunner have redirected the standard output and error buffers for the spawned process so I could display the contents of those buffers in the NAntRunner interface. Before 0.4, I would wait until the process finished and then get the entire contents of the buffer:

Class NAntProcess
    …
    Public ReadOnly Property StandardError() As String
        Get
            Return Me._SystemProcess.StandardOutput.ReadToEnd
        End Get
    End Property

    Public ReadOnly Property StandardError() As String
        Get
            Return Me._SystemProcess.StandardOutput.ReadToEnd
        End Get
    End Property
    …
End Class

When I started using the verbose switch, the NAnt process hung because the process generated much more output than before (imagine that, “verbose” equals “more”), and the output buffers filled up before the process finished. So, I had to change how I was grabbing the output messages off.

The VelocityReviews.com link above mentions BeginOutputReadLine, so I did some more digging and came up with this MDSN article which shows how to pull these messages off asynchronously:

http://msdn.microsoft.com/en-us/library/system.diagnostics.process.beginoutputreadline.aspx

This involved handling the OutputDataReceived and ErrorDataReceived handlers for the process, and then appending the messages as they were generated using a pair of StringBuilder objects. The modifications looked like the following (only the relevant portions of the NAntProcess class are shown):

Class NAntProcess
    Private _StandardOutputBuilder As StringBuilder
    Private _ErrorOutputBuilder As StringBuilder
    Public Sub New(ByVal NAntExecutablePath As String, _
                   ByVal NewScriptToRun As String, _
                   ByVal NewTargetFramework As String, _
                   ByVal NewBuildTarget As String, _
                   ByVal ShouldEnableVerboseMessages As String, _
                   ByVal NewOtherArgs As String)
        …

        ‘ Configure the asynchronous output collection
        Me._StandardOutputBuilder = New StringBuilder
        Me._StandardOutputBuilder.AppendLine(Me._NAntExecutable & ” “ & Me.GetArgumentsForNAnt)
        Me._StandardOutputBuilder.AppendLine()
        Me._StandardOutputBuilder.AppendLine()
        Me._ErrorOutputBuilder = New StringBuilder
        AddHandler Me._SystemProcess.OutputDataReceived, AddressOf Me.StandardOutputHandler
        AddHandler Me._SystemProcess.ErrorDataReceived, AddressOf Me.ErrorOutputHandler
    End Sub

    Private Sub StandardOutputHandler(ByVal sendingProcess As Object, ByVal outLine As DataReceivedEventArgs)
        If Not String.IsNullOrEmpty(outLine.Data) Then
            Me._StandardOutputBuilder.AppendLine(outLine.Data)
        End If
    End Sub

    Private Sub ErrorOutputHandler(ByVal sendingProcess As Object, ByVal outLine As DataReceivedEventArgs)
        If Not String.IsNullOrEmpty(outLine.Data) Then
            Me._ErrorOutputBuilder.AppendLine(outLine.Data)
        End If
    End Sub

    Public ReadOnly Property StandardOutput() As String
        Get
            Return Me._StandardOutputBuilder.ToString
        End Get
    End Property

    Public ReadOnly Property StandardError() As String
        Get
            Return Me._ErrorOutputBuilder.ToString
        End Get
    End Property
End Class

After I got this implemented, I realized that I wouldn’t need the verbose switch to cause this problem. In theory, the “regular” output from a very involved NAnt script could cause the buffers to fill up. It’s just a fluke that I hadn’t hit this up to this point.

Advertisements

September 7, 2009 - Posted by | Tools and Toys, Visual Studio/.NET

2 Comments

  1. any great solutions for manager unhandled exceptions when buffer overrun or outofmemory exception ? thanks

    Comment by handler | May 7, 2010

    • Can you be a little more specific? Buffer overruns and out of memory exceptions can be caused by a number of things. In general, “handling” them usually means not writing more to the buffer than it can hold (the solution described in this post) or detecting when it happens and dealing with the exception gracefully.

      “Gracefully” in this case could be reporting the error, or rolling back and restarting the process, but neither really get you past the original problem – how to write X bytes of data when you have a buffer (or memory space) that can only hold Y bytes.

      Comment by markegilbert | May 7, 2010


Sorry, the comment form is closed at this time.

%d bloggers like this: