Tuesday, August 7, 2007

PGP Pipeline Component

Recently I was required to perform some PGP encryption and decryption of files. Realizing this was going to require a custom Pipeline Component, off I went to Google to find one. Hey, why reinvent the wheel.

I'm not certain why Microsoft didn't put one in place with the release of BTS 2006, but who am I to judge. :)

The ones that kept popping up:

GnuPG
This one works very well. I actually used GnuPG when creating a pipeline component for another project. However, it is a command line program and requires installation and key management before it works. While I knew this would get the job done, I wanted to use something that could be automatically deployed from machine to machine.

Pro BizTalk 2006
Since I have not purchased this book, I don't have access to the code. Again, I don't want to buy something I know I can do.
Don't get me wrong, I'm not opposed to buying code, but it needs to make sense. And in this case, it didn't. Especially since I have done this before and knew the job could be performed in the allotted timeframe. Also, I have nothing against this book. It comes highly reviewed. I'm just a little put-off about buying another BizTalk book. I waited and waited for the official BTS 2004 book to come out and was very disappointed. Ok, so I need to get over it. I will probably buy a BTS 2006 book eventually. I just don't know which one. [end rant]

Bouncy Castle Crypto
A co-worker pointed me in the direction of the Bouncy Castle C# API. They give you the DLL as well as the source code. The only problem I ran in to was that the DLL was not strongly named. Once I resolved that issue I was off and running. This gave me the ability to easily deploy from server to server. The only thing I had to do was GAC the DLL and copy the keys.

Here is the code for the PGP Pipeline component. I did not distribute the necessary crypto.dll, so you will need to get it from Bouncy Castle. Remember, you will need to get the source in order to strongly name the assembly.

[UPDATED - 7/27/2007] - I have updated the code and source with version 1.1.

Link to readme.txt: readme.txt
Link to dll: BAJ.BizTalk.PipelineComponent.PGP.dll
Link to source code: PGP.zip

Notes:

  • I can't get the stupid icon to appear for whatever reason. If you spot my error, please let me know so I can fix it.
  • The project has a Post Build Event that will copy the DLL to C:\Program Files\Microsoft BizTalk Server 2006\Pipeline Components.
    • You may need to change the path based on your environment.
    • You will receive a build failure if you have a project open that references that file. Also, you may need to stop your BizTalk host before compilation once you have deployed and used the component.
  • The code assumes you already have the necessary Public and/or Private keys. I did not include a way to generate the pair.
  • Be careful when testing. I chased around this error message for almost an hour before I realized what was happening: Could not find file 'C:\Temp\testfile.txt'.
    • The PGP Pipeline Component expects the encrypted version of the file and the decrypted version of the file to have the same name with the exception of the .PGP extension. If you encrypt somefile.txt, it becomes somefile.txt.pgp. If you then rename somefile.txt.pgp to differentfile.txt.pgp, the crypto.dll writes the decrypted file to somefile.txt.
    • Because I wasn't sure what the end-user's desired outcome should be (error if the filenames don't match, always use the filename of the message, etc.) I left it alone and created a DecryptFileAsStream() method that does not create a file, but returns the decrypted content as a Stream.
  • I had thought about adding a property to specify the temporary location where the file is written during encryption/decryption but have not had time to add it. Currently, it will place the temporary files in C:\Windows\System32.
    • Ok, so now I have added it. :)

Feedback welcomed.

33 comments:

Shaun said...

Brian,
Great pipeline component, appreciate the effort you've gone to, to create it and present it here.

I was able to get things working but had some difficulties when it came to the Bouncy Castle component. As you explained, the component needs to be strongly named and therefore you need to compile the source. I'm using 2.0 which successfully upgraded the source code.

However, compiling the source proved difficult because the Bouncy Castle implementation appears to have a heavy dependency on NUnit. While I'm a big fan of NUnit and unit testing in general, I don't fancy having yet another dependency to deal with when I have no need to unit test someone else's component code.

I solved the issue by installing and referencing NUnit but I'd be really interested if others had the same issue but solved it differently and could do without NUnit altogether.

Thanks again for a great component.

Brian said...

Shaun,

I'm glad you are able to use the PGP Pipeline Component. Sorry for the mess with NUnit. I didn't realize the dependency so I must have had it from a previous project. You are the first to mention NUnit. Others have had some difficulty with the Bouncy Castle part tho.

Anyone else experience any issues?

Let me know your feedback and any improvements I can make, or you have made.

Leo said...

I've just excluded the Test folder from crypto.csproj, removed the NUnit references and worked like a charm!

Anonymous said...

Brian,
The component is great. Like Shaun I installed NUnit to get it all working. One question I have though is 'how do i get it working with encryped and SIGNED files?' I have 2 pgp files one encrypted. and one encrypted and signed. The encrypted one works fine but the encrypted and signed one just gives me the error 'Org.BouncyCastle.Bcpg.OpenPgp.PgpException: message is not a simple encrypted file - type unknown' I'm guesssing it's complaining because it doesn't know about the signature details, but I can't figure out how to refer to them.

Any ideas would be great - thanks again.
Ben

Brian said...

By looking at your message, I am guessing your are tripping on the type of message being processed. I am looking at the code, and in PGPWrapper.cs on line 183, I had to put in some special checks to see if the file was signed before I can decrypt it. There may be a sequencing issue that your document contains that I have not yet encountered.

I would recommend running the code in debugger and put a breakpoint at the line referenced above. Stepping through the code should provide some assistance in troubleshooting the problem.

I would be interested in hearing what you do to resolve the issue.

David said...

After downloading the Bouncy Castle source, examining it, compiling, and running the NUnit tests I ...

- create a .snk to sign the assembly
- changed the build option options for test code and data from:
>(test/src/*) Compile -> None
>(test/data/*) EmbeddedResource -> None
- removed NUnit references

I then built the "Release" solution and got a much smaller .dll (1.1M vs 2.6M) to deploy and no "extra" log4j or NUnit dependencies.

For .NET projects I prefer to separate NUnit tests into their own project within a solution to avoid the extraneous dependences but it is sure hard to discredit the Bouncy Castle Crypto authors. I very much appreciate their efforts.

Eugene Mayevski said...

If problems with BouncyCastle appear, you can try SecureBlackbox for BizTalk, which includes OpenPGP adapter as well.

Daniel said...

Hi Brian,

Is there any chance you could send me a copy of the crypto.dll file, as I would need a reference to it to build the project?
I could not get through to Bouncy Castle on that one, which is why I am asking you for it...

Much appreciation,

Brian said...

Daniel - I believe I used the v1.1 version of the BouncyCastle code. However, I see they have recently posted v1.4. The link to their source code download is here. You will need to download the code, strongly name it, compile it, and then reference your version. If you still have difficulties, please post back and I will move to "Plan B".

Daniel said...

Hi Brian,

Thanks for the tip.

I downloaded the source code from Bouncy Castle (v 1.4), strongly named the assembly and compiled it to generate my version of the crypto.dll file. I then referenced the dll in your project PGP, and built it, so now I have this pipeline DLL installed under the path "C:\Program Files\Microsoft BizTalk Server 2006\Pipeline Components". Do I need to install this assembly in the GAC, in order to consume it in a BizTalk project in VS 2005?

Regards.

Daniel said...

Brian,

I tried to build/deploy the PGPExample project and got the following error pointing to both the Decrypt.btp and Encrypt.btp objects:

Component 'PGP Pipeline Component [BAJ]' properties validation failed. Invalid Operation:

No other error details are provided and I did not modify any of the Pipeline Component Properties.

Can you tell me what is causing these build errors?

Daniel said...

Brian:

I loaded the PGP Pipeline component into the GAC and started a new BizTalk project. In that project, I added the component into the Toolbox area when creating a Receive Pipeline object. When I dragged and dropped the component into the Designer view of the Receive Pipeline (in the Decode stage), I get a Microsoft Visual Studio error stating:

"Pipeline component Load() method failed on IPersistPropertyBag implementation: 'PGP Pipeline Component [BAJ]'"

Any ideas on how to resolve this issue?

Thanks,

Daniel.

Sunil said...

Brain,

I am trying to use your Pipeline Component. I downloaded the code for building the crypto DLL and also Strong named it. Then I have downloaded your code for the Pipeline component and built the pipeline component. When I start using the pipeline component after copying it into appropriate location and also copied the DLLs to GAC I am running into the error "Component 'PGP Pipeline Component [BAJ]' properties validation failed. Invalid Operation:" Would please let me know if I have missed any Step. Appreciate your Help

Brian said...

Sunil,

It sounds like you are missing some of the property values, specifically the Operation. Make sure that you select either Encrypt or Decrypt from the Pipeline component properties window.

HTH -- Brian

Sunil said...

Thanks Brian. Now I am able to Compile. One more Question regarding generating the Keys; do we have to purchase any Software to Generate the Private & Public Keys Or is there any freeware available. Thanks for the Help

Brian said...

Sunil,

Glad things are working for you now. I used GnuPG v1.4.7 to generate my keys.

Sunil said...

Could you please give me the link where I can Download the Software. Thanks for the Help

Brian said...

http://www.gnupg.org/

Nick said...

Brian-

Thanks for the component. Any chance that you're going to add a EncryptAndSign() method?

Thanks.

Anonymous said...

The Bouncy Castle site download links are broken links. Any other place to get that source?

Brian said...

If you go to the Bouncy Castle C# Home page, you will be able to download the latest source.

Wayne Magnum said...

Hello Brian,

I'm receiving the following error when i try to test the components out in BizTalk.

There was a failure executing the receive pipeline: "PGPExample.Pipelines.Receive.Encrypt, PGPExample, Version=1.0.0.0, Culture=neutral, PublicKeyToken=698baaeee7a4ff40" Source: "PGP Pipeline Component [BAJ]" Receive Port: "Brian_PGPPipelineComponent_ReceivePort" URI: "E:\Decrypted\*.txt" Reason: Path cannot be the empty string or all whitespace.

Can you point me the right direction oh what path it's is referring to.

Basically i'm dropping a file into the Decrypted folder to be picked up and encrypted and placed into the encrypted folder.

Thank you,
-Wayne

Wayne Magnum said...

Hi Brian,

I was able to resolve the problem...It was a typo on my end plus also the information in the Readme.txt file helped out.

Great Post.

Thank you,
-Wayne Magnum

Anonymous said...

Got the following when i ran test encrypted doc through inbound pipeleine

Reason: Could not load file or assembly 'crypto, Version=1.4.0.0, Culture=neutral, PublicKeyToken=3c6cd05f3c105626' or one of its dependencies. The system cannot find the file specified.

Mark said...

Now i get this:

Reason: Org.BouncyCastle.Bcpg.OpenPgp.PgpPublicKeyRing found where PgpSecretKeyRing expected

In pipeline config:
PrivateKeyfile = C:\GnuPG\"vendorname".asc
PublicKeyfile = C:\GnuPG\"merchant".asc

I an not clear on the Private/Public designation.

GnuPG generated one and I imported one from CommerceHub. or Am I missing Something?

Anonymous said...

Brian:

Thanks for sharing your work with us. Would you repost the code please? The link to the the PGP.zip download file appeared to be broken. Appreciate your help.

Brian said...

I am still working on getting a new FTP location. In the mean time, you can email me and I will send you the code.

jonesb321 [at] gmail [dot] com

Raja said...

Brian, Lemme get pgp code. Please it would be great helpful.

Thanks in Advance,
Raja
rajatpk@hotmail.com
raja.kumaravel@in.ispace.com

rajatpk said...

Thanks for your code.
I generated the key which is having private & public key info using GNU Privacy assistant.
so i could assign only one key for private & public key path on BizTalk Encrypt & Decrypt properties.
Now i am getting error as "unknown object in stream 21". Plese guide me.
Thanks,
Raja

Kannan said...

I am getting the same error.. "Unknown object in stream 21". Any thoughts on how to fix this?

Stuart said...

This has been massively useful, thanks for posting it. Got it working nicely with a simple FILE adapter in Biztalk 2010.

I installed and started using the BLogical SFTP adapter which works very nicely in Biztalk 2010 on its own.

However, when I set up the SFTP adapter and set it to use the custom pipeline which uses this PGP decryption component at the same time, things start to go wrong and I was getting weird errors from the encrypt sections of code within of the PGP component when I clearly had selected decrypt. I put some tracing into the PGP source code which identified that the properties entered in the compoent (i.e. private key, public key, encrypt/decrypt etc) were not being loaded from the PropertyBag and were being set to default values or null. The logging has confirmed that the implemented method which sets the parameters "void IPersistPropertyBag.Load(IPropertyBag propertyBag, int errorLog)" is being invoked, suggesting that the IPropertyBag object itself has not picked up the values that have been set from within the component/administrator.

Has anyone come across this before? Could there be a bit of code missing within the custom SFTP adapter that needs to supply the correct ProperyBag object. Or maybe the PropertyBag needs to be initiated from within the Pipeline Component in some other way? Any help appreciated.

Anitha said...

Brian,

I have an error when trying to "Encrypt" in the send pipeline I added the pipeline component dll (BAJ.BizTalk.PipelineComponent.PGP.dll) and choose the items to tool box, drag and dropped in the encode section, gac the dll. I deployed
Below is the error
A message sent to adapter "FILE" on send port "SndQuestExtractFFport" with URI "C:\TestLocations\Quest\QuestEligibility_%MessageID%.txt" is suspended.
Error details: There was a failure executing the send pipeline: "QuestEligExt.Pipelines.SendQuestEligFF, QuestEligExt.Pipelines, Version=1.0.0.0, Culture=neutral, PublicKeyToken=69ca41c0c461ffc2" Source: "PGP Pipeline Component [BAJ]" Send Port: "SndQuestExtractFFport" URI: "C:\TestLocations\Quest\QuestEligibility_%MessageID%.txt" Reason: Object reference not set to an instance of an object.

Can you let me know why I am getting this error

Anonymous said...

Brain,

Thanks for putting this out.
I am looking at your example, however I am unable to find the binding file.
Can you tell me where I can get that.

Thanks