skip to content
Relatively General .NET

How not to read a string from an UTF-8 stream

by Gérald Barré

posted on: September 06, 2021

This post is part of the series 'Strings in .NET'. Be sure to check out the rest of the blog posts of the series!String comparisons are harder than it seemsHow to correctly count the number of characters of a stringCorrectly converting a character to lower/upper caseHow not to read a string from an

Database and Always-Valid Domain Model

by Vladimir Khorikov

posted on: August 31, 2021

Today, we’ll talk about an important question: how does the application database fit into the concept of Always-Valid Domain Model? In other words, is the database part of the always-valid boundary or should you consider it an external system and validate all data coming from it?

Exploring the NuGet client libraries

by Gérald Barré

posted on: August 30, 2021

We often use NuGet through Visual Studio or dotnet CLI, but you can also use the same using your own application. In this post, I'm going to show multiple examples of using the .NET client library for NuGet.To start using the NuGet client library, you need to add a reference to the NuGet packages:c

From MVC to Minimal APIs with ASP.NET Core 6.0

by Ben Foster

posted on: August 28, 2021

ASP.NET 6.0 introduces an alternative way to build HTTP APIs, using the aptly named “Minimal APIs”. This post provides a step-by-step guide on how to translate traditional MVC concepts to this new approach.

Cascading retries and the sulky applications

by Oren Eini

posted on: August 27, 2021

I recently run into a bit of code that made me go: Stop! Don’t you dare going this way! The reason that I had such a reaction for the code in question is that I have seen where such code will lead you, and that is not anywhere good. The code in question? This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters Show hidden characters public static List<T> LoadData<T>(string connectionName, string sqlQuery, Dictionary<string, object> parameters, string failedMessage) { var dynamicParameters = new DynamicParameters(parameters); List<T> results = null; int waitTime = Int32.Parse(System.Configuration.ConfigurationManager.AppSettings["WaitTime"].ToString()); int retry = 0; while (retry < 5) { try { using (var connection = new SqlConnection(ConnectionString(connectionName))) { results = connection.Query<T>(sqlQuery, dynamicParameters, commandTimeout: 15 * 60).ToList<T>(); } break; } catch (Exception ex) { retry++; Logger.Log.Error($"{failedMessage}, failed on retry {retry}", ex); System.Threading.Thread.Sleep(waitTime * retry); } } return results; } view raw opps.cs hosted with ❤ by GitHub This is a pretty horrible thing to do to your system. Let’s count the ways: Queries are happening fairly deep in your system, which means that you’re now putting this sort of behavior in a place where it is generally invisible for the rest of the code. What happens if the calling code also have something similar? Now we got retries on retries. What happens if the code that you are calling has something similar? Now we got retries on retries on retries. You can absolutely rely on the code you are calling to do retries. If only because that is how TCP behaves. But also because there are usually resiliency measures implemented. What happens if the error actually matters. There is no exception throw in any case, which means that important information is written to the log, which no one ever reads. There is no distinction of the types of errors where retry may help and where it won’t. What is the query has side effects? For example, you may be calling a stored procedure, but multiple times. What happens when you run out of retries? The code will return null, which means that the calling code will like fail with NRE. What is worst, by the way, is that this piece of code is attempting to fix a very specific issue. Being unable to reach the relevant database. For example, if you are writing a service, you may run into that on reboot, your service may have started before the database, so you need to retry a few times to the let the database to load. A better option would be to specify the load order of the services. Or maybe there was some network hiccup that you had to deal with? That would sort of work, and probably the one case where this will work. But TCP already does that by resending packets, you are adding this again and it is building up to be a nasty case. When there is an error, your application is going to sulk, throw strange errors and refuse to tell you what is going on. There are going to be a lot of symptoms that are hard to diagnose and debug. To quote Release It!: Connection timeouts vary from one operating system to another, but they’re usually measured in minutes! The calling application’s thread could be blocked waiting for the remote server to respond for ten minutes! You added a retry on top of that, and then the system just… stops. Let’s take a look at the usage pattern, shall we? This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters Show hidden characters foreach(var item in Query<Orders>(conStr, "SELECT * FROM Orders where UserId = @userId", new {["UserId"] = userID), "load queries")) { // do something here. } view raw stop.cs hosted with ❤ by GitHub That will fail pretty badly (and then cause a null reference exception). Let’s say that this is a service code, which is called from a client that uses a similar pattern for “resiliency”. Question – what do you think will happen the first time that there is an error?  Cascading failures galore. In general, unknown errors shouldn’t be handled locally, you don’t have a way to do that here. You should raise them up as far as possible. And yes, showing the error to the user is general better than just spinning in place, without giving the user any feedback whatsoever.

Converting PFX format to PEM via OpenSSL programmatically

by Oren Eini

posted on: August 26, 2021

I run into a task that I needed to do in Go, given a PFX file, I needed to get a tls.X509KeyPair from that. However, Go doesn’t have support for PFX. RavenDB makes extensive use of PFX in general, so that made things hard for us. I looked into all sorts of options, but I couldn’t find any way to manage that properly. The nearest find was the pkcs12 package, but that has support for only some DER format, and cannot handle common PFX files. That was a problem.Luckily, I know how to use OpenSSL, but while there are countless examples on how to use OpenSSL to convert PFX to PEM and the other way around, all of them assume that you are using that from the command line, which isn’t what we want. It took me a bit of time, but I cobbled together a one off code that does the work. The code has a strange shape, I’m aware, because I wrote it to interface with Go, but it does the job. This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters Show hidden characters #include <string.h> #include <openssl/pkcs12.h> #include <openssl/pem.h> #include <openssl/bio.h> #include <openssl/err.h> #include <openssl/x509.h> void init_errors(){ ERR_load_crypto_strings(); } int get_pem_size(void * pem) { char * buf; int len = BIO_get_mem_data(pem, & buf); return len; } void copy_pem_to(void * pem, void * dst, int size) { char * buf; int len = BIO_get_mem_data(pem, & buf); memcpy(dst, buf, len > size ? size : len); } void free_pem(void * pem) { BIO_free(pem); } char * pfx_to_pem(void * data, long size, char * pwd, void ** key, void ** crt) { char * rc = NULL; BIO * bio = NULL; PKCS12 * p12 = NULL; EVP_PKEY * pkey = NULL; X509 * cert = NULL; STACK_OF(X509) * ca = NULL; BIO * key_bio = NULL; BIO * crt_bio = NULL; bio = BIO_new_mem_buf(data, size); if (!bio) { rc = "Unable to allocate memory buffer"; goto cleanup; } p12 = d2i_PKCS12_bio(bio, NULL); if (!p12) { rc = "Unable to read certificate"; goto cleanup; } if (!PKCS12_parse(p12, pwd, & pkey, & cert, & ca)) { rc = "Unable to parse certificate"; goto cleanup; } key_bio = BIO_new(BIO_s_mem()); if (!key_bio) { rc = "Out of memory, cannot create mem BIO for key"; goto cleanup; } if (!PEM_write_bio_PrivateKey(key_bio, pkey, NULL, NULL, 0, NULL, NULL)) { rc = "Failed to write PEM key output"; goto cleanup; } * key = key_bio; crt_bio = BIO_new(BIO_s_mem()); if (!crt_bio) { rc = "Out of memory, cannot create mem BIO for cert"; goto cleanup; } if (!PEM_write_bio_X509(crt_bio, cert)) { rc = "Failed to write PEM crt output"; goto cleanup; } * crt = crt_bio; goto success; cleanup: if (key_bio) BIO_free(key_bio); if (crt_bio) BIO_free(crt_bio); success: if (bio) BIO_free(bio); if (p12) PKCS12_free(p12); if (pkey) EVP_PKEY_free(pkey); if (cert) X509_free(cert); return rc; } view raw pfx_to_pem.c hosted with ❤ by GitHub Now, from Go, I can run the following: This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters Show hidden characters //#cgo CFLAGS: "-IC:/Program Files/OpenSSL-Win64/include" //#cgo LDFLAGS: "-LC:/Program Files/OpenSSL-Win64/lib" -llibcrypto // #include "pfx.c" import "C" var initialized bool // from: // https://github.com/spacemonkeygo/openssl/blob/c2dcc5cca94ac8f7f3f0c20e20050d4cce9d9730/init.go func errorFromErrorQueue() string { if initialized == false { initialized = true C.init_errors() } var errs []string for { err := C.ERR_get_error() if err == 0 { break } errs = append(errs, fmt.Sprintf("%s:%s:%s", C.GoString(C.ERR_lib_error_string(err)), C.GoString(C.ERR_func_error_string(err)), C.GoString(C.ERR_reason_error_string(err)))) } return fmt.Sprintf("SSL errors: %s", strings.Join(errs, "\n")) } func pfx_to_pem(pfx []byte) (key_buf []byte, crt_buf []byte, err error) { var key *C.void var crt *C.void rc := C.pfx_to_pem(unsafe.Pointer(&pfx[0]), C.long(len(pfx)), nil, (*unsafe.Pointer)(unsafe.Pointer(&key)), (*unsafe.Pointer)(unsafe.Pointer(&crt))) if rc != nil { err = errors.New(C.GoString(rc) + "\n" + errorFromErrorQueue()) return } defer C.free_pem(unsafe.Pointer(key)) defer C.free_pem(unsafe.Pointer(crt)) size := C.get_pem_size(unsafe.Pointer(key)) key_buf = make([]byte, int(size)) C.copy_pem_to(unsafe.Pointer(key), unsafe.Pointer(&key_buf[0]), size) size = C.get_pem_size(unsafe.Pointer(key)) crt_buf = make([]byte, int(size)) C.copy_pem_to(unsafe.Pointer(crt), unsafe.Pointer(&crt_buf[0]), size) return } view raw pfx.go hosted with ❤ by GitHub As you can see, most of the code is there to manage error handling. But you can now convert a PFX to PEM and then pass that to X509keyPair easily.That said, this seems just utterly ridiculous to me. There has got to be a better way to do that, surely.