This article takes you to master two common authentication methods of OBS

Absrtact: This article will show you two common authentication methods of OBS - Header carry signature and URL carry signature.

OBS provides REST (representative state transfer) style API, which enables you to call through HTTP/HTTPS request. Before calling the OBS API, you need to know the authentication method of OBS. This article will take you through two common authentication methods of OBS - Header carry signature and URL carry signature.

1. Carry signature calculation in Header

Official website link: https://support.huaweicloud.com/api-obs/obs_04_0010.html

1.1 calculation principle and method of signature

Schematic diagram

computing method

(1). Construct the request string (StringToSign).

The request string is constructed as follows:

StringToSign =

HTTP-Verb + "\n" +

Content-MD5 + "\n" +

Content-Type + "\n" +

Date + "\n" +

CanonicalizedHeaders + CanonicalizedResource

(2). The results of the first step are UTF-8 encoded.

(3). Use SK to calculate HMAC-SHA1 signature for the result of the second step.

(4). The result of the third step is Base64 encoded to obtain the signature.

The signature is in the following form (28 bit BASE64 encoded string):

JONydLd9zpf+Eu3IYiUjNmukHN0= 

Calculation example

Example: you need to get the object log under the bucket "OBS test" Conf object ACL, how to construct the request and calculate the signature?

1. First, specify the fields of StringToSign:

Request method: GET;

Request MD5: null

Content type: empty

Request time: Tue, 28 Jul 2020 06:29:47 GMT (i.e. 14:29:47 Beijing time, July 28, 2020)

CanonicalizedHeaders: empty

Canonicalized resource: / OBS test / log conf? acl

2. Construct the request string StringToSign as follows:

StringToSign = '''GET
 
 
Tue, 28 Jul 2020 06:29:47 GMT
/obs-test/log.conf?acl'''

3. According to the signature algorithm, the StringToSign is calculated by HMAC-SHA1 and then encoded by BASE64 to obtain the signature result: xYlcrwT9jSaCtY0OnBE01OBR+aA=

1.2 implementation of signature calculation

Take Python calculation signature code as an example for reference:

 import hashlib  
 import hmac  
 import binascii  
 from datetime import datetime  
    
  # Verification information  
  AK = 'Yours access_key_id'  
 SK = 'Yours secret_access_key_id'  
  
# Specify HTTP method, optional: GET/PUT/DELETE/POST/OPTIONS  
httpMethod = "GET"  
 
# Specify the header of the request: content type and Content-MD5  
contentType = ""  
 conten**5 = ""  
  
 # Use the datetime library to generate the time. If you need to customize the request time, please keep the format consistent  
 date = datetime.utcnow().strftime('%a, %d %b %Y %H:%M:%S GMT')  
   
 # Fill in canonicalizedHeaders  
# canonicalizedHeaders = "x-obs-acl:public-read"  
 # canonicalizedHeaders = "x-obs-acl:public-read\n"+'x-obs-storage-class:WARM\n'  
 canonicalizedHeaders = ""  
   
 # Fill in CanonicalizedResource  
 # CanonicalizedResource = "/BucketName/ObjectName"  
 # CanonicalizedResource = "/BucketName/ObjectName?acl"  
 # CanonicalizedResource = "/"
 CanonicalizedResource = "/BucketName/"  
  
# Generate StringToSign  
 canonical_string = httpMethod + "\n" + conten**5 + "\n" + contentType + "\n" + date + "\n" + canonicalizedHeaders + CanonicalizedResource  
 
 # Calculate the signature and encode BASE64  
 hashed = hmac.new(SK.encode('UTF-8'), canonical_string.encode('UTF-8'), hashlib.sha1)  
 encode_canonical = binascii.b2a_base64(hashed.digest())[:-1].decode('UTF-8')  
  
 # Print StringToSign for verification in case of problems  
 print(canonical_string)  
   
 # Print signature  
 print(encode_canonical)  

Example of C language signature algorithm:

Please refer to https://obs-community.obs.cn-north-1.myhuaweicloud.com/sign/signature_c.zip , download the C language signature calculation code example, where:

The interface for calculating the signature is included in sign H header file.

The sample code for calculating the signature is in main C in the document.

Visual signature calculation tool:

You can also calculate the signature through the visual signature calculation tool provided by OBS.

Tool link:

https://obs-community.obs.cn-north-1.myhuaweicloud.com/sign/header_signature.html

explain:

1. Canonicalized headers: indicates the OBS request header field in the HTTP request header field, that is, the header field with "x-obs -" as the dropout, such as "x-obs-date, x-obs-acl, x-obs-meta - *";

a. All characters of keywords in the request header field should be converted to lowercase. When multiple fields need to be added, all fields should be sorted from small to large according to the dictionary order of keywords;

b. When adding request header fields, if there are fields with duplicate names, they need to be merged. For example, x-obs-meta-name:name1 and x-obs-meta-name:name2, the values of the duplicate name fields (here name1 and name2) need to be separated by commas and merged into x-obs-meta-name:name1 and name2;

c. Keywords in the request header field in the header field are not allowed to contain non ASCII codes or unrecognizable characters; The value in the request header field is not recommended to use non ASCII code or unrecognizable characters. If non ASCII code or unrecognizable characters must be used, the client needs to encode and decode by itself. URL coding or Base64 coding can be used, and the server will not decode;

d. When the request header field contains meaningless spaces or table keys, it needs to be discarded. For example: x-obs-meta-name:name (with a meaningless space in front of name), which needs to be converted to: x-obs-meta-name:name;

e. A new line is required at the end of each request header field.

2. Canonicalized resource refers to the OBS resource specified in the HTTP request. The construction method is as follows:

< bucket name + object name > + [sub resource]

a. Access OBS through the bucket bound custom domain name. The bucket name is represented by the custom domain name, which is "/ obs.ccc.com/object", where "obs.ccc.com" is the bucket bound custom domain name. If there is no object name, such as enumerating buckets, it is "/ obs.ccc.com /";

b. For the scenario of accessing OBS through a custom domain name not bound by a bucket, it is "/ bucket/object". If there is no object name, such as enumerating buckets, it is "/ bucket /". If there is no bucket name, it is "/";

c. If there are sub resources, add the sub resources, for example? acl,? logging.

3. If it is necessary to use the temporary AK/SK+SecurityToken method to calculate the signature, the method of calculating the signature shall be consistent, but the "x-obs-security-token:..." field needs to be added to the header field.

4. See the description at the end of the text for the method of calculating Content-MD5.

5. For the code of calculating signature in other languages, please refer to the SDK of the corresponding language for details:

https://support.huaweicloud.com/sdkreference-obs/obs_02_0001.html

1.3 common problems

1. When accessing OBS, there is an error: Signature Does Not Match

There are two main possibilities of signature mismatch:

a. If you don't use the correct AK/SK, you can check whether the SK you use to calculate the signature and the AK you carry when sending the request are correct and match;

b. The StringToSign constructed when you calculate the signature does not match the StringToSign calculated by the server according to the received HTTP request. You can check the StringToSign returned by the server and compare it with the StringToSign used for local signature calculation.

As shown in the figure below, the StringToSign returned by the server is restored by receiving the HTTP request. You can analyze the reason why your signature calculation fails by comparing your local StringToSign with the HTTP request you sent to the server.

2. Error when accessing OBS: Request has expired

In this case, please check whether the Date you carry is correct. In order to ensure the timeliness of the request, the Date header field you carry must be within 15 minutes from the time of the server (the server is UTC time). If you carry the x-obs-date header field, you need to check whether the time of x-obs-date is within 15 minutes from the time of the server.

2. Carry signature in URL

OBS service supports users to construct a URL for a specific operation. This URL will contain user AK, signature, validity period, resources and other information. Anyone who gets this URL can perform this operation. After receiving this request, OBS service considers that the request is to sign the URL and the user is performing the operation. For example, construct the URL of a download object with signature information. The person who gets the corresponding URL can download the object, but the URL is only valid within the expiration time specified by Expires. The signature carried in the URL is mainly used to enable others to use the pre signed URL for identity authentication and perform predefined operations without providing the Secret Access Key to others.

Official website link https://support.huaweicloud.com/api-obs/obs_04_0011.html

2.1 calculation principle and method of signature

Schematic diagram

computing method

1. Construct the request string (StringToSign).

The request string is constructed as follows:

StringToSign =

HTTP-Verb + "\n" +

Content-MD5 + "\n" +

Content-Type + "\n" +

Date + "\n" +

CanonicalizedHeaders + CanonicalizedResource

2. UTF-8 code the results of the first step.

3. Use SK to calculate HMAC-SHA1 signature for the result of the second step.

4. Base64 encode the result of the third step to obtain the signature.

The signature is in the following form (28 bit BASE64 encoded string):

JONydLd9zpf+Eu3IYiUjNmukHN0=

The signature calculation method carried in the URL is the same as that carried in the Header, but the Date needs to be replaced with a UNIX timestamp.

The URL with signature is as follows:

obs-ycytest.obs.cn-north-1.myhuaweicloud.com/?AccessKeyId=YN97UCJEKF2ALJ44AHAN&Expires=1575452568&Signature=0wG/GF7XgmOatCFhwHJh0J6NrtQ=

The corresponding StringToSign is

GET

1575452568
/obs-ycytest/

The specific meanings of the parameters carried in the URL are shown in the following table:

Calculation example

Example: you need to get the object log under the bucket "OBS test" Conf object ACL, how to construct the request and calculate the signature?

1. First, specify the fields of StringToSign:

Request method: GET;
request MD5: empty
Content-Type: empty
 Request time: 1595918661 (i.e. July 28, 2020 Beijing time):44:21)
Custom header field( CanonicalizedHeaders): empty
 Standardized resources( CanonicalizedResource):/obs-test/log.conf?acl

2. Construct the request string StringToSign as follows:

StringToSign = '''GET
 
 
1595918661
/obs-test/log.conf?acl'''

3. According to the signature algorithm, the StringToSign is calculated by HMAC-SHA1 and then encoded by BASE64 to obtain the signature result: lcyw1ffmjv5m + ms0xennrqjlag=

According to the calculation results, the URL with signature can be generated by splicing the URL as follows:

obs-test.obs.myhuaweicloud.com/log.conf?AccessKeyId=xxx&acl&Expires=1595918661&Signature= lLcYw1fFMJv5m+MS0XenNrqJlag=

2.2 implementation of signature calculation

When a signature is carried in the URL, the corresponding signature can be calculated by replacing the Date with the UNIX timestamp, so the corresponding code will not be repeated.

If it is necessary to use the temporary AK/SK+SecurityToken method to calculate the signature, the method of calculating the signature shall be consistent, but the "? x-obs-security-token =..." field shall be added to the corresponding canonicalized resource, and the calculated signature must be URL encoded. The code of using temporary AK/SK+SecurityToken to calculate signature is as follows:

  import hashlib  
  import hmac  
  import binascii  
  import urllib.request  
    
  AK = 'Input Your AccessKeyId'  
 SK = 'Input Your SecretKeyId'  
 Token = 'Input Your SecurityToken'  
    
 httpMethod = "GET"  
 contentType = ""  
 Conten**5 = ''  
 date = '1594972984'  
 canonicalizedHeaders = ''  
 CanonicalizedResource = "/messageflow/flowengine.tar.gz" + "?x-obs-security-token=" + Token  
 canonical_string = httpMethod + "\n" + Conten**5 + "\n" + contentType + "\n" + date + "\n" + canonicalizedHeaders + CanonicalizedResource  
 hashed = hmac.new(SK.encode('UTF-8'), canonical_string.encode('UTF-8'), hashlib.sha1)  
 encode_canonical = binascii.b2a_base64(hashed.digest())[:-1].decode('UTF-8')  
   
 url= 'messageflow.obs.myhuaweicloud.com/flowengine.tar.gz?x-obs-security-token={}&Expires={}&AccessKeyId={}&Signature={}'.format(Token, date, AK, urllib.request.quote(encode_canonical))  
 print(url)  

explain:

1. When calculating the signature, Date represents a UNIX timestamp;

2. If you want to use the URL to carry the signature in the browser and generate the predetermined URL, do not use "Content-MD5", "content type" and "CanonicalizedHeaders" to calculate the signature, otherwise the browser cannot carry these parameters, and the signature error will be prompted after the request is sent to the server.

3. Calculation method of Content-MD5

3.1 calculation method of Content-MD5

Taking the message content "0123456789" as an example, the method of calculating the Content-MD5 of the string is described in detail below.

1. First calculate the binary array (128 bits) encrypted by MD5.

2. base64 encode the binary array (instead of 32-bit string encoding).

Take Python as an example:

>>> import base64,hashlib
>>> hash = hashlib.md5()
>>> hash.update("0123456789".encode('utf-8'))
>>> base64.b64encode(hash.digest())
'eB5eJF1ptWaXm4bijSPyxw=='
Note: hash.digest(),Calculate the binary array (128 bits).
>>> hash.digest()
'x\x1e^$]i\xb5f\x97\x9b\x86\xe2\x8d#\xf2\xc7'

3.2 implementation of Content-MD5 calculation

Take the Python calculation file MD5 code as an example for reference:

  import os  
 import base64  
 import hashlib  
   
  
  def md5_file_encode_by_size_offset(file_path=None, size=None, offset=None, chuckSize=None):  
     if file_path is not None and size is not None and offset is not None:  
         m = hashlib.md5()  
        with open(file_path, 'rb') as fp:  
             CHUNKSIZE = 65536 if chuckSize is None else chuckSize  
             fp.seek(offset)  
            read_count = 0  
             while read_count < size:  
                read_size = CHUNKSIZE if size - read_count >= CHUNKSIZE else size - read_count  
                data = fp.read(read_size)  
                 read_count_once = len(data)  
                if read_count_once <= 0:  
                    break 
                 m.update(data)  
                 read_count += read_count_once  
         return base64.b64encode(m.digest()).decode()  
   
   
 file_path = r'Input Your File Path'  
 size = os.path.getsize(file_path)  
 Conten**5 = md5_file_encode_by_size_offset(file_path=file_path, size=size, offset=0)  
 print(Conten**5)  

3.3 common problems

A common error is to base64 encode the calculated 32-bit string directly.

# hash.hexdigest() calculates the visible 32-bit string encoding.
>>> import base64,hashlib
>>> hash = hashlib.md5()
>>> hash.update("0123456789".encode('utf-8'))
>>> hash.hexdigest()
'781e5e245d69b566979b86e28d23f2c7'

#The result of base64 encoding the wrong MD5 value:

>>> base64.b64encode(hash.hexdigest())
'NzgxZTVlMjQ1ZDY5YjU2Njk3OWI4NmUyOGQyM2YyYzc='

Click follow to learn about Huawei's new cloud technology for the first time~

Tags: REST url header

Posted by ethridgt on Tue, 24 May 2022 18:42:58 +0300