by Andrew Lock
posted on: April 20, 2021
In this post I describe how to hide the warning generated by the dotnet CLI when working with end-of-life frameworks like .NET Core 3.0 (netcoreapp3.0)…
by Gérald Barré
posted on: April 19, 2021
Your continuous integration script often looks like dotnet build mysolution.sln and dotnet test mysolution.sln. In my case, I have a solution that contains multiple class libraries and a few WPF projects. I want to create NuGet packages and also test projects on Windows, Linux, and Mac. However, yo
by Oren Eini
posted on: April 15, 2021
Voron is RavenDB’s storage engine. It is how we store data, keep transactions and in generally get a lot of our abilities. In this post, I want to talk about the way RavenDB manages disk space internally. Voron uses a single data file to do its work, the data file is divided into 8KB pages, like so:Voron uses eager disk allocations to reserve disk space from the operating system. Each time the space inside the file runs out, Voron will double the size of the file. That last until the file size reaches 2GB, after which RavenDB will grow by 1GB at a time. This behavior ensures that Voron gives the underlying file system enough information to provide the database with a continuous range of disk space. In other words, we grab disk space in large chunks to avoid fragmentation of the data file. What happens when you delete data, however? Voron mark the free space in its free list and will use that space before it will allocate more disk space from the operating system.Why aren’t we releasing the disk space back to the operating system? The simplest reason is that it isn’t an just the data at the end of the file that is freed. In fact, like in the image above, free and busy segments are interwoven in the file. We can’t just truncate the file.Internal references inside of RavenDB make use of the position data inside the file, so just moving the data won’t help. Instead, you have to compact the data. That forces us to re-write the entire database layout from scratch and fixes those references. That is an offline operation, however. For the most part, it doesn’t actually matter. RavenDB will use the internal free space as needed, so it isn’t like it is actually lost. One feature that we are considering for version 6.0 of RavenDB is hole punching. That means that we’ll make use of advanced file system API to free the disk space allocated to RavenDB even mid file.On Linux, that means using FALLOC_FL_PUNCH_HOLE. On Windows, that means using FSCTL_SET_ZERO_DATA.That will have the advantage of freeing disk space back to the operating system without needing user intervention. In particular, that is going to make it so a user that delete data to free disk space see the free space reflected in the OS metrics.There are problems with this approach, however. First, the size of the file remains the same, which leads to interesting questions. Consider:Second, this defeats the purpose of wanting to optimize disk allocations. If we free disk space in this manner, when we get it back, it may no longer be continuous on the disk. That said, it is not that big a problem in the days of SSDs and NVMes as it was at the time of the rotational hard disk. Then, you may get into a very bad situation in which Voron tries to use disk space that it had allocated mid file (but was already freed) but it can’t, because the disk is full. Right now, this is simply an impossible error, with hole punching, we need to consider how to deal with this.
by Ardalis
posted on: April 14, 2021
When you're writing unit tests for a method, it's a good idea to test various failure conditions ("sad paths") in addition to testing the…Keep Reading →
by Andrew Lock
posted on: April 13, 2021
In this post I show how you can view the overridden values from configuration for debugging purposes.…
by Oren Eini
posted on: April 12, 2021
I’m doing some cloud work, and I am working based off the official documentation, trying to automate the creation of an AWS Lambda. In order to allow me to quickly iterate, I’m basically creating the entire thing from scratch each time. I have the following code: aws iam create-role --role-name $AWS_ROLE --assume-role-policy-document file://trust-policy.json aws iam attach-role-policy --role-name $AWS_ROLE --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole aws lambda create-function --function-name $FUNC_NAME --zip-file fileb://lambda.zip --handler lambda_function.lambda_handler --runtime python3.8 --role $ARN_ROLE So far, so good, and exactly like it shows in the docs. But if you’ll run it as a script, it will fail with: An error occurred (InvalidParameterValueException) when calling the CreateFunction operation: The role defined for the function cannot be assumed by Lambda. If I re-run the exact same command, however, it works properly. There is this interesting command, which indicates that roles are using eventual consistency: aws iam wait role-exists --role-name $AWS_ROLE Except… that this doesn’t work. It looks like there is some additional delay between creating the role, validating that it was created and when it is actually available for Lambda to be used. After looking around and feeling like a fool, I added a sleep for 10 seconds to the script, and the problem went away. I’m posting this for posterity sake and in the hope that someone can tell me that there is a better way. For now, I think I need a shower.
by Gérald Barré
posted on: April 12, 2021
This website is hosted on a static website hosting provider and I use Cloudflare CDN to improve the performance. These 2 services allow me to host this website for free even if I got more and more visitors. However, I do not have lots of analytics. For instance, I want to know where the user comes
by Ben Foster
posted on: April 10, 2021
In this post we explore the new IAuthorizationMiddlewareResultHandler interface in .NET 5 that enables you to customize the HTTP response when authorization fails.
by Oren Eini
posted on: April 09, 2021
We just published a white paper on RavenDB performance vs. Couchbase performance in a real customer scenario.I had to check the results three times before I believed them. RavenDB is pretty awesome, but I had no idea it was that awesome. The data set was reasonably big, 1.35 billion docs and the scenario we present is a real world one based on production load. Some of the interesting details:RavenDB uses 1/3 of the disk space that Couchbase uses, but stores 3 times as much data.Operationally, RavenDB just worked, Couchbase needed 6 times the hardware to just scrape by. A single failure in Couchbase meant at least 15 – 45 minutes for the node to recover. Inducing failures in RavenDB brought the node back up in a few seconds. For queries, we pitted a Couchbase cluster with 96 cores and 384 GB RAM against single RavenDB node running on a Raspberry PI. RavenDB on the Pi was able to sustain better latencies at the 99 percentile handling twice as much load as Couchbase is able.There are all sort of other goodies in the white paper and we went pretty deep into the overall architecture and impact of the difference design decisions.As usual, we welcome your feedback.
by Oren Eini
posted on: April 08, 2021
You can hear me speaking at the Angular Show about using document database from the point of view of full stack or front end developers.In this episode, panelists Brian Love, Jennifer Wadella, and Aaron Frost welcome Oren Eini, founder of RavenDB, to the Angular Show. Oren teaches us about some of the key decisions around structured vs unstructured databases (or SQL vs NoSQL in hipster developer parlance). With the boom of document-driven unstructured databases, we wanted to learn why you might choose this technology, the pitfalls and benefits, and what are the options out there. Of course, Oren has a bit of a bias for RavenDB, so we'll learn what RavenDB is all about and why it might be a good solution for your Angular application.
