Accessing Azure Storage Account File Shares from C# Applications
Azure Storage File Shares client library for .NET – Version 12.4.0
https://docs.microsoft.com/en-us/dotnet/api/overview/azure/storage.files.shares-readme
Azure File Shares offers fully managed file shares in the cloud that are accessible via the industry standard Server Message Block (SMB) protocol. Azure file shares can be mounted concurrently by cloud or on-premises deployments of Windows, Linux, and macOS. Additionally, Azure file shares can be cached on Windows Servers with Azure File Sync for fast access near where the data is being used.
Azure SDK Latest Releases
https://azure.github.io/azure-sdk/releases/latest/index.html
SDKs and tools for managing and interacting with Azure services
https://azure.microsoft.com/en-us/downloads/
Programmatically manage and interact with Azure services.
The Azure SDKs are collections of libraries built to make it easier to use Azure services from your language of choice. These libraries are designed to be consistent, approachable, diagnosable, dependable, and idiomatic. See the latest releases, documentation, and design guidelines.
Microsoft Azure Storage Explorer
https://docs.microsoft.com/en-us/azure/vs-azure-tools-storage-manage-with-storage-explorer
Microsoft Azure Storage Explorer is a standalone app that makes it easy to work with Azure Storage data on Windows, macOS, and Linux. In this article, you’ll learn several ways of connecting to and managing your Azure storage accounts.
Create a C# Console App (.NET Framework)
I will create a Console App (.NET Framework) to demonstrate the access to an Azure Storage Account File Share.
Install Azure Storage File Shares client library for .NET from NuGet
Open the NuGet Package Manager and browse for the Azure Storage Files Shares Package as follows.
From PowerShell
dotnet add package Azure.Storage.Files.Shares
As you can see, NuGet added a bunch of new references to my project from which the first three Azure.Core, AzureStorage.Common and Azure.Storage.Files.Shares are the ones we actually need now.
Upload files to a share
Now I will create an new class which will contain our code to access the Azure Storage Account File Share and to upload a file into this share.
As I will use the App.config file to store the connectionString to our Azure Storage Account, we first need to add a further reference to our project.
Use Visual C# to store and retrieve custom information from an application configuration file
https://docs.microsoft.com/en-us/troubleshoot/dotnet/csharp/store-custom-information-config-file
To retrieve the key/value pairs from App.config file, we need to add the System.Configuration assembly as follows.
Inside our new class Azure.cs, we first add a few new using directives, so that we not to have write the whole namespaces in front of our commands.
using System.Configuration;
using Azure;
using Azure.Storage.Files.Shares;
using System.IO;
Next we add a key/value pair inside the appSettings tag with our connectionString to the Storage Account in the App.config file.
You will find the Connection String of your Storage Account under Settings – Access keys in the Azure Portal and your Storage Account.
<appSettings> <add key="StorageConnectionString" value="DefaultEndpointsProtocol=https;AccountName=<your account name>;AccountKey=<your account key>;EndpointSuffix=core.windows.net" /> </appSettings>
Now we can add the code to access the Storage Account inside our new class Azure.cs.
Therefore we create a new public method named UploadToAzure with two string parameters, the first sFilename is the name the file should be named after the upload in azure and the second sPath is the local path + filename from where we want to upload the file to azure.
// sFilename = Name of the file how should appear in Azure Storage Account after upload // sPath = local path to file we want to upload including the filename itself public void UploadToAzure(string sFilename, string sPath) { // retrieve the key/value pair from the App.config file string connectionString = ConfigurationManager.AppSettings.Get("StorageConnectionString"); // Name of the share, directory, and file string shareName = "braintesting"; string dirName = "Administration"; string fileName = sFilename; // Get a reference from our share ShareClient share = new ShareClient(connectionString, shareName); // Get a reference from our directory - directory located at root level ShareDirectoryClient directory = share.GetDirectoryClient(dirName); // Get a reference to our file and upload it to azure ShareFileClient file = directory.GetFileClient(fileName); using (FileStream stream = File.OpenRead(sPath)) { file.Create(stream.Length); file.UploadRange( new HttpRange(0, stream.Length), stream); } }
Now finally we had to create in the Program.cs class an instance of our new Azure.cs class and trigger the UploadToAzure(string sFilename, string sPath) method.
For this demonstration I will pass the two string parameters directly to the method.
class Program { static void Main(string[] args) { Azure az = new Azure(); az.UploadToAzure("sample.xlsx", "C:\Temp\attachments\sample.xlsx"); } }
Upload Files to a Subdirectory
If you want to upload a file into a subdirectory of your share, you have to reference the ShareDirectoryClient class to this subdirectory.
Previously we just referenced a directory at root level of our share with
ShareDirectoryClient directory = share.GetDirectoryClient(dirName);
Now we have to reference all levels to our subdirectory C1
https://stacbraintesting.file.core.windows.net/braintesting/Administration/A1/B1/C1
// Get a reference from our root level directory
ShareDirectoryClient directory = share.GetDirectoryClient(dirName);
// Get a reference to a subdirectory C1
directory = directory.GetSubdirectoryClient(“A1”);
directory = directory.GetSubdirectoryClient(“B1”);
directory = directory.GetSubdirectoryClient(“C1”);
ShareDirectoryClient directory now holds a reference to our subdirectory C1
// sFilename = Name of the file how should appear in Azure Storage Account after upload // sPath = local path to file we want to upload including the filename itself public void UploadToAzure(string sFilename, string sPath) { string connectionString = ConfigurationManager.AppSettings.Get("StorageConnectionString"); // Name of the share, directory, and file string shareName = "braintesting"; string dirName = "Administration"; string fileName = sFilename; // Get a reference from our share ShareClient share = new ShareClient(connectionString, shareName); // Get a reference from our directory - directory located on root level ShareDirectoryClient directory = share.GetDirectoryClient(dirName); // Get a reference to a subdirectory not located on root level directory = directory.GetSubdirectoryClient("A1"); directory = directory.GetSubdirectoryClient("B1"); directory = directory.GetSubdirectoryClient("C1"); // Get a reference to our file ShareFileClient file = directory.GetFileClient(fileName); // Max. 4MB (4194304 Bytes in binary) allowed const int uploadLimit = 4194304; // Upload the file using (FileStream stream = File.OpenRead(sPath)) { file.Create(stream.Length); if (stream.Length <= uploadLimit) { file.UploadRange( new HttpRange(0, stream.Length), stream); } else { int bytesRead; long index = 0; byte[] buffer = new byte[uploadLimit]; // Stream is larger than the limit so we need to upload in chunks while ((bytesRead = stream.Read(buffer, 0, buffer.Length)) > 0) { // Create a memory stream for the buffer to upload MemoryStream ms = new MemoryStream(buffer, 0, bytesRead); file.UploadRange(new HttpRange(index, ms.Length), ms); index += ms.Length; // increment the index to the account for bytes already written } } }
Traverse the share
If your want to get all folder and filenames located in your share, you can traverse the share to get a full list of all folders and files in this share.
Here all folder and filenames will be added to the 2-n tuple list named folders and be returned from this method GetFolders().
public List<Tuple<string,bool>> GetFolders() { // Azure Storage Account Connection String and share name string connectionString = ConfigurationManager.AppSettings.Get("StorageConnectionString"); string shareName = "braintesting"; // reference a new list 2-tuple named folders which will include our directories from our share braintesting in our Azure Storage Account var folders = new List<Tuple<string, bool>>(); // Get a reference from our share ShareClient share = new ShareClient(connectionString, shareName); // Track the remaining directories to walk, starting from the root var remaining = new Queue<ShareDirectoryClient>(); remaining.Enqueue(share.GetRootDirectoryClient()); while (remaining.Count > 0) { // Get all of the next directory's files and subdirectories ShareDirectoryClient dir = remaining.Dequeue(); foreach (ShareFileItem item in dir.GetFilesAndDirectories()) { // Add the directory names and filenames to our list 2-tuple folders.Add(new Tuple<string, bool>(item.Name, item.IsDirectory)); // Keep walking down directories if (item.IsDirectory) { remaining.Enqueue(dir.GetSubdirectoryClient(item.Name)); } } } return folders; }
Download a File from a share
To download a file from our Azure Storage Account share we only need to modify the above UploadToAzure() method a little bit.
Depending on downloading a file from a folder on root level of your share or from a subdirectory, you need to reference your directory appropriate as above for the upload.
The part to download a file is as follows
// Download the file ShareFileDownloadInfo download = file.Download(); using (FileStream stream = File.OpenWrite(sPath)) { download.Content.CopyTo(stream); }
So our complete DownloadFromAzure() method is
public void DownloadFromAzure(string sFilename, string sPath) { string connectionString = ConfigurationManager.AppSettings.Get("StorageConnectionString"); // Name of the share, directory, and file string shareName = "braintesting"; string dirName = "Administration"; string fileName = sFilename; // Get a reference from our share ShareClient share = new ShareClient(connectionString, shareName); // Get a reference from our directory - directory located on root level ShareDirectoryClient directory = share.GetDirectoryClient(dirName); // Get a reference to a subdirectory not located on root level directory = directory.GetSubdirectoryClient("A1"); directory = directory.GetSubdirectoryClient("B1"); directory = directory.GetSubdirectoryClient("C1"); // Get a reference to our file ShareFileClient file = directory.GetFileClient(fileName); // Download the file ShareFileDownloadInfo download = file.Download(); using (FileStream stream = File.OpenWrite(sPath)) { download.Content.CopyTo(stream); } }
The DownloadFromAzure() method in our Azure.cs class, we also call from our Main method in the Program.cs class, and will pass the two parameters, first the name of the file we want to download and second the local path + filename in where the file should be downloaded and named into.
class Program { static void Main(string[] args) { Azure az = new Azure(); az.DownloadFromAzure("sample.xlsx", "C:\Temp\sample.xlsx"); } }
Download with the Async APIs
The file download from an Azure Storage Account share supported both synchronous and asynchronous APIs.
To use the Async API we need to modify the above code a little bit.
In order to use the await operator, we must mark our DownloadFromAzure() method with the async and Task modifier as follows.
// adding the async modifier public async Task DownloadFromAzure(string sFilename, string sPath)
Further we have to change the ShareFileClient.Download method into the ShareFileClient.DownloadAsync method as follows.
// Download the file asynchron with the Async APIs ShareFileDownloadInfo download = await file.DownloadAsync(); using (FileStream stream = File.OpenWrite(sPath)) { await download.Content.CopyToAsync(stream); }
And finally we have to call the DownloadFromAzure() method also from an async method instead so far from our static void Main() entry point method from the Program.cs class.
Therefore I added an async void method named DownloadFile() in the Program.cs class, which is then calling the public async task DownloadFromAzure() method in the Azure.cs class and will be itself called by the static void Main() method.
So the complete Program.cs class looks as follows.
class Program { static void Main(string[] args) { DownloadFile(); Thread.Sleep(5000); } static async void DownloadFile() { Azure az = new Azure(); await az.DownloadFromAzure("sample.xlsx", "C:\Temp\sample.xlsx"); } }
You may wonder about the Thread.Sleep(5000) in the Main method?
So what would happen without pause the Main method after triggering the download?
Because we invoke both asynchronous download methods with the await keyword, they herself will immediately return control back to the caller of the method, in our case of this simple console app, the Main method, which will have no further instructions and therefore ends the execution of the program immediately, before the separate thread with the download will be finished.So finally the download fails!
Mainly the asynchronous download make sense in GUI applications, in order to not block the current thread with the download and therefore avoiding that the GUI not responds.
And the complete async DownloadFromAzure() method looks as follows.
public async Task DownloadFromAzure(string sFilename, string sPath) { string connectionString = ConfigurationManager.AppSettings.Get("StorageConnectionString"); // Name of the share, directory, and file string shareName = "braintesting"; string dirName = "Administration"; string fileName = sFilename; // Get a reference from our share ShareClient share = new ShareClient(connectionString, shareName); // Get a reference from our directory - directory located on root level ShareDirectoryClient directory = share.GetDirectoryClient(dirName); // Get a reference to a subdirectory not located on root level directory = directory.GetSubdirectoryClient("A1"); directory = directory.GetSubdirectoryClient("B1"); directory = directory.GetSubdirectoryClient("C1"); // Get a reference to our file ShareFileClient file = directory.GetFileClient(fileName); //// Download the file asynchronous with the Async APIs ShareFileDownloadInfo download = await file.DownloadAsync(); using (FileStream stream = File.OpenWrite(sPath)) { await download.Content.CopyToAsync(stream); } }
More about Asynchronous programming
Asynchronous programming with async and await
https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/concepts/async/
Asynchronous Programming Model (APM)
https://docs.microsoft.com/en-us/dotnet/standard/asynchronous-programming-patterns/asynchronous-programming-model-apm
async (C# Reference)
https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/async
Troubleshooting
Error when you write more than 4 MB of data to Azure Storage: Request body is too large
https://docs.microsoft.com/en-us/troubleshoot/azure/general/request-body-large
Assume that you try to write more than 4 megabytes (MB) of data to a file on Azure Storage, or you try to upload a file that’s larger than 4 MB by using the SAS url (REST API) or HTTPS PUT method. In this scenario, the following error is returned:
Code : RequestBodyTooLarge
Message: The request body is too large and exceeds the maximum permissible limit.
RequestId:<ID>
MaxLimit : 4194304
There’s a 4-MB limit for each call to the Azure Storage service. If your file is larger than 4 MB, you must break it in chunks. For more information, see Azure Storage scalability and performance targets.
Solution:
break your file into chunks not larger than 4 MB (4194304 bytes in binary)
// Max. 4MB (4194304 Bytes in binary) allowed const int uploadLimit = 4194304; using (FileStream stream = File.OpenRead(sPath)) { file.Create(stream.Length); // Stream is not above the Limit of 4 MB upload complete file if (stream.Length <= uploadLimit) { file.UploadRange( new HttpRange(0, stream.Length), stream); } else { int bytesRead; long index = 0; byte[] buffer = new byte[uploadLimit]; // Stream is larger than the limit so we need to upload in chunks while ((bytesRead = stream.Read(buffer,0,buffer.Length)) > 0) { // Create a memory stream for the buffer to upload MemoryStream ms = new MemoryStream(buffer, 0, bytesRead); file.UploadRange(new HttpRange(index, ms.Length), ms); index += ms.Length; // increment the index to the account for bytes already written } } }
Links
Azure SDK Latest Releases
https://azure.github.io/azure-sdk/releases/latest/index.html
SDKs and tools for managing and interacting with Azure services
https://azure.microsoft.com/en-us/downloads/
Azure.Storage.Files.Shares Namespace
https://docs.microsoft.com/en-us/dotnet/api/azure.storage.files.shares
Microsoft Azure Storage Data Movement Library (2.0.0)
https://github.com/Azure/azure-storage-net-data-movement
The Microsoft Azure Storage Data Movement Library designed for high-performance uploading, downloading and copying Azure Storage Blob and File. This library is based on the core data movement framework that powers AzCopy.
ShareDirectoryClient.GetSubdirectoryClient(String) Method
https://docs.microsoft.com/en-us/dotnet/api/azure.storage.files.shares.sharedirectoryclient.getsubdirectoryclient
Create File
https://docs.microsoft.com/en-us/rest/api/storageservices/create-fileAsynchronous Programming Model (APM)
https://docs.microsoft.com/en-us/dotnet/standard/asynchronous-programming-patterns/asynchronous-programming-model-apm
Get started with Microsoft Azure Storage Explorer
https://docs.microsoft.com/en-us/azure/vs-azure-tools-storage-manage-with-storage-explorer