PayPal REST API OAuth2 Authentication

There’s a lot of confusion about exactly what login/password (or clientID/clientSecret) is to be used for obtaining an OAuth2 access token for PayPal REST API calls.

PayPal uses simple HTTP Basic authentication (protected by a TLS connection) to obtain an access token.   Using the Chilkat Rest library, your application would provide the credentials in this way (in pseudo-code)

rest.SetAuthBasic("CLIENT_ID", "API_SECRET");

So… where exactly does one get the CLIENT_ID and CLIENT_SECRET???

Go to https://developer.paypal.com/developer/accounts/

Click on the “Profile” link for one of your Sandbox accounts.  Then go to the “API Credentials” tab.  You’ll see the following.
(Note: You should have first defined/created a REST app within your PayPal account.  You’ll see it here.  In my case, the name of my REST App is “Chilkat”.  Yours will be whatever you named it.)

Next, click on your REST App.  You’ll see this:

The Client ID and API Secret are the values that should be passed to rest.SetAuthBasic.

Calling a REST POST API and Parsing JSON/XML Response

This blog post describes a general procedure for writing code that calls an HTTP/HTTPS POST REST API and parses the JSON or XML response.

Step 1: Form the CURL Command

A POST request in the form of a CURL command will look like this:

curl -X POST https://test-api.service.hmrc.gov.uk/organisations/vat/123456789/returns \
-H "Accept: application/vnd.hmrc.1.0+json"  \
-H "Content-Type: application/json" \
-H "Authorization: Bearer hmrc_app_server_token" \
-d '{
  "periodKey": "#001",
  "vatDueSales": 100.00,
  "vatDueAcquisitions": 100.00,
  "totalVatDue": 200,
  "vatReclaimedCurrPeriod": 100.00,
  "netVatDue": 100,
  "totalValueSalesExVAT": 500,
  "totalValuePurchasesExVAT": 500,
  "totalValueGoodsSuppliedExVAT": 500,
  "totalAcquisitionsExVAT": 500,
  "finalised": true
}'

Some notes:

  • Header fields are added using the “-H” option. Add the header fields and values as required by your particular REST API.
  • If OAuth2 authentication is required, the OAuth2 access token is provided in an “Authorization: Bearer …” header field.
  • If HTTP Basic authentication is required, which means providing a username/password, then use the –user option like this:
    --user username:password
    
  • If the request body contains JSON or XML, then query params are added in the URL and should appear just as they would in a browser address bar.  For example: https://test-api.service.hmrc.gov.uk/organisations/vat/123456789/returns?param1=value1&param2=value2
  • If the Content-Type is to be application/x-www-form-urlencoded, then params are specified using “-d” options. In this case, the Content-Type does not need to be explicitly specified.  For example:
    curl -X POST http://mws.amazonservices.com/Feeds/2009-01-01 \
      -d "AWSAccessKeyId=AWS_ACCESS_KEY_ID" \
      -d "Action=CancelFeedSubmissions" \
      -d "FeedSubmissionIdList.Id.1=1058369303" \
      -d "FeedTypeList.Type.1=_POST_PRODUCT_DATA_" \
      -d "FeedTypeList.Type.2=_POST_PRODUCT_PRICING_DATA_" \
      -d "MWSAuthToken=MWS_AUTH_TOKEN" \
      -d "Marketplace=ATExampleER" \
      -d "SellerId=MWS_SELLER_ID" \
      -d "SignatureMethod=HmacSHA256" \
      -d "SignatureVersion=2" \
      -d "Timestamp=CURRENT_DATE_TIME" \
      -d "Version=2009-01-01" \
      -d "Signature=MWS_SIGNATURE"
    
  • The JSON or XML body is specified in a “-d” option.  Enclose the full JSON or XML document in single quotes as shown above.

Step 2. If the REST API POST returns JSON or XML, get a representative sample of the JSON or XML response. For example:

{
  …  whatever JSON is returned goes here …
}

Step 3. Generate Code

Go to the online tool at http://tools.chilkat.io/curl.cshtml and generate code in any of 29 different programming languages. Paste the CURL command and respresentative JSON/XML response into the text boxes, and select the type of response (JSON, XML, Text, or Binary). If the POST returns just a status code, such as 201, and no response body, then select “Text” as the type of response.

Then click “Generate Code” and your sample code is generated. It generates code to create the JSON or XML request body, send the request, and to parse the JSON or XML response.   (If you already have the JSON/XML request body composed by some other means, then just ignore the generated code that creates the JSON/XML.)

Calling a REST GET API and Parsing JSON/XML Response

This blog post describes a general procedure for writing code that calls an HTTP/HTTPS GET REST API and parses the JSON or XML response.

Step 1: Form the CURL Command

A GET request in the form of a CURL command will look like this:

curl -X GET https://test-api.service.hmrc.gov.uk/organisations/vat/123456789/obligations \
-H "Accept: application/vnd.hmrc.1.0+json"  \
-H "Authorization: Bearer hmrc_app_server_token"

Some notes:

  • Header fields are added using the “-H” option.  Add the header fields and values as required by your particular REST API.
  • If OAuth2 authentication is required, the OAuth2 access token is provided in an “Authorization: Bearer …” header field.
  • If HTTP Basic authentication is required, which means providing a username/password, then use the –user option like this:
    --user username:password
    
  • Query params are added in the URL and should appear just as they would in a browser address bar.

Step 2. If the REST API GET returns JSON or XML, get a representative sample of the JSON or XML response.  For example:

{
  "obligations": [
    {
      "start": "2017-01-01",
      "end": "2017-03-31",
      "due": "2017-05-07",
      "status": "F",
      "periodKey": "18A1",
      "received": "2017-05-06"
    },
    {
      "start": "2017-04-01",
      "end": "2017-06-30",
      "due": "2017-08-07",
      "status": "O",
      "periodKey": "18A2"
    }
  ]
}

Step 3.  Generate Code

Go to the online tool at http://tools.chilkat.io/curl.cshtml and generate code in any of 29 different programming languages.  Paste the CURL command and respresentative JSON/XML response into the text boxes, and select the type of response (JSON, XML, Text, or Binary).  If the GET returns just a status code, such as 201, and no response body, then select “Text” as the type of response.

Then click “Generate Code” and your sample code is generated.  It generates code to send the request and to parse the JSON or XML response.

Chilkat v9.5.0.75 Release Notes

The v9.5.0.74 release notes are available here: Chilkat v9.5.0.74 Release Notes

v9.5.0.75 Release Notes:

  • Important:  This release accidentally breaks Amazon MWS (not AWS) authentication.  If using MWS, send email to support@chilkatsoft.com for a hotfix, or revert back to v9.5.0.73.
  • Smartcard/USB Tokens Support for Smartcard and USB tokens is much improved. Note: At this point, smartcards and USB tokens are supported on Windows operating systems only.
  • XmlDSigGen Many improvements and fixes to XML digital signature generation and verification…
  • Cert Added the Cert.SmartCardPin property.
  • Amazon/AWS Fixed problems with non-us-ascii chars in x-amz-meta-* headers
  • AuthUtil Fixed AuthUtil.WalmartSignature for POST/PUT operations.
  • Imap Fixed: The QuotaRoot method returned invalid JSON in some cases.
  • PFX/P12 Added the ability to open .pfx/.p12 files where two different passwords are required: one for integrity and the other for the private keys. See the example at https://www.example-code.com/csharp/pfx_multiple_passwords.asp
  • Cert Fixed the ValidFrom and ValidTo properties for some certificates (a rare problem, the vast majority of certs exhibited no problems with these properties).
  • ZipCrc Added the CrcString, CrcSb, and CrcBd methods for greater flexibility in calculating CRC32 checksums.
  • Cert Added the HashOf method to return the encoded hash of a particular part of a certificate.
  • Http Added the CreateTimestampRequest and VerifyTimestampReply methods to provide the ability to send RFC 3161 timestamp requests to a TSA (Timestamp Authority).
  • SFtp Fixed the LastReadNumBytes method which always returned 0.
  • Http Fixed certain URL and percent encoding problems for paths and query params.
  • PrivateKey Added the LoadAnyFormat method to load a private key from any text or binary format.
  • Rest Fixed auto-reconnect when the connection is thought to exist, but found to not exist when sending the request.
  • XmlDSigGen Added the empty-string option for the canonMethod argument to the AddSameDocRef, AddObjectRef, and AddExternalXmlRef methods. This makes it possible to add a reference with no transformations.
  • Ssh Added the Ssh.ClientPort property.
  • Rest Can set the expectedStatus argument of the SetResponseBodyStream to a negative number to indicate a range of expected status codes. (See the online reference documentation.)
  • CkString Fixed: The saveToFile method did not write a file with no BOM (Byte Order Mark) when passing “no-bom-utf-8” for the charsetEncoding.
  • XmlDSigGen Added the “TransformSignatureXPath” keyword to the Behaviors property.
  • Http Added the ResumeDownloadBd method.
  • Http Fixed receiving XML responses where the charset is not indicated in the Content-Type response header, but is available in the XML declarator contained in the response body.
  • SFtp Downloading to a file (DownloadFileByName) will check to see if the local target file already exists. If so, it will try to delete it prior to downloading. If it is not possible to delete, then the DownloadFileByName will return failure before trying to download the file.
  • Ssh Fixed the Ssh.HostKeyFingerprint property for ECDSA keys.
  • Http Fixed problem that occurred for request sent with: “Accept-Encoding: gzip, deflate”
  • XmlDSigGen Added the SignedInfoId property.
  • XmlDSigGen Added the KeyInfoId property.
  • XmlDSigGen Added the SetRefIdAttr method.
  • encoding Added the “hexlower” encoding. “hexlower” may be used instead of “hex” to get the hexadecimal encoding using lowercase a-f (“hex” produces uppercase A-F).
  • JsonObject Added the InsertNewArray and InsertNewObject methods.
  • PFX/P12 Fixed a problem with not properly recognizing OID 1.2.840.113549.1.1.10 in some circumstances.
  • Email Fixed a problem with the SetSigningCert and SetSigningCert2 methods.

Bd Methods Prevent Copying Huge Amounts of Data

Question: 

One use is for unzip a resource embed into an assembly (.dll). The resource is approximately 224MB in size.   The program sometimes throws an out-of-memory exception.   The exception is thrown by Chilkat.Zip.OpenFromMemory and OpenFromByteData methods.

How can this problem be solved?

Answer:

First, this is truly an out-of-memory condition.  The two possible solutions are (1) to allow your application to use more memory (not sure if this is possible, but you might look here for information on the topic:  https://www.codeproject.com/Articles/483475/Memory-Limits-in-a-NET-Process ) or (2) to use less memory.

The OpenBd method can be used to significantly reduce memory usage.   The problem with OpenFromMemory (and OpenFromByteData) is that the contents of the zip are passed in a .NET managed byte array.  The data must be marshaled from managed memory to unmanaged, and this causes the full 224MB to exists twice in memory (in managed memory within your .NET application, and in unmanaged memory within the underlying native Chilkat C++ implementation).

A better solution is to use a Chilkat.BinData object to load the contents of the file, and then call zip.OpenBd.  This way you’re just passing a handle to the BinData object instead of 224MB.  The zip data exists only once in memory.

*** In general, Chilkat is incrementally adding “Bd” and “Sb” methods for those methods in Chilkat classes that can potentially pass large amounts of binary or text data.  The “Sb” methods pass a Chilkat.StringBuilder object.  The same concept applies:  Instead of marshaling a huge string in the method call, the application can pass a StringBuilder object which contains the text.

Using “Bd” and “Sb” methods is better for other programming languages too, especially anything using the ActiveX because strings in ActiveX/COM are passed as BSTR’s (utf-16) and binary byte data is passed in Variants.  The overhead of conversion/marshaling can be eliminated by using the Bd and Sb methods.

 

Stripe Webhooks?

Question from User:   Is there some sample code for Stripe Webhooks in classic ASP? I looked around, and didnt see anything.  Specifically, validating the signatures https://stripe.com/docs/webhooks/signatures

Answer:  A Webhook is an HTTP POST that will get sent to your server when a particular event occurs.   You would write an ASP page to receive the HTTP POST.  The ASP code to receive the HTTP POST is just straight ASP (no Chilkat involved).

You’ll write ASP code to get the POST’s data. If the POST was an application/json Content-Type, then the body of the HTTP POST will contain JSON.   If the POST was an application/x-www-form-urlencoded Content-Type, then you’ll have name/value params.  The Stripe online developer documentation should tell you about format and data contained in each event’s POST.

Once you’ve accessed/extracted the POST’s data, then you might use Chilkat to help in processing it — such as for the JSON parsing or signature validation.  To help answer questions about signature validation, I would first need to see a sample of the POST data for that particular case.

 

Etsi, XAdES-BES, XAdES-EPES, FacturaE, Electronic Invoicing, etc.

Chilkat v9.5.0.75 will include features to make it easy to generate XAdES-BES and XAdES-EPES signatures for electronic invoicing and tax reporting for many countries.

The goal is to provide an easy and inexpensive solution for these complicated requirements.  Chilkat has been working with customers in Spain, Italy, Hungary, India, Brazil, Costa Rica, and elsewhere to get things working smoothly.  (Contact Matt at support@chilkatsoft.com for more information.)

The vision is to offer an online tool that generates the solution (source code) given a sample of already-signed XML.  This tool is live and functioning at https://tools.chilkat.io/xmlDsigGen.cshtml. The tool can generate code in any of approximately 30 different programming languages.  The tool will be updated and refined as Chilkat solves issues and challenges brought by the requirements of new customers.   XML digital signatures can be tricky, fragile, brittle, and different implementations for checking the validity of a signature can be quirky, have undocumented requirements, etc.  The idea is to build into the code generation tool the expertise for knowing the particular needs of different authorities, so you don’t have to spend weeks of frustration in getting things to work.

As time passes, and as more expertise is embodied within the code generation tool, I believe Chilkat can reach the point where it “just works”.

 

How to Encrypt with no Padding (encrypted output size equals input size)

Block encryption algorithms, such as AES, will produce output that is a multiple of the algorithm’s block size. For AES, the output is a multiple of 16 bytes. However, this is for the typically used cipher modes “CBC” (Cipher Block Chaining) and “ECB” (Electronic Cookbook).

There are other cipher modes that turn a block algorithm into a stream cipher. See https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation

Namely, the cipher modes are:

  • Counter Mode (ctr)
  • Cipher Feedback (cfb)
  • Output Feedback (ofb)

In C++, for example, any of the above modes can be used with AES to produce output that is exactly the same size as the input.
For example


    CkCrypt2 crypt;

    crypt.put_CryptAlgorithm("aes");
    // Use Counter mode.  Other stream modes are "cfb" and "ofb".
    crypt.put_CipherMode("ctr");
    crypt.put_KeyLength(128);
    crypt.SetEncodedKey("000102030405060708090A0B0C0D0E0F","hex");
    crypt.SetEncodedIV("A0A1A2A3A4A5A6A7A8A9AAABACADAEAF","hex");

    const unsigned char plainText[] = { 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23 };

    CkByteData data;
    data.append2(plainText,23);

    CkByteData encData;
    crypt.EncryptBytes(data,encData);

    printf("size of encrypted data = %d\n",encData.getSize());
    const unsigned char *pEncData = encData.getData();

IMPORTANT: The output of encryption consists of bytes that resemble random data, where each byte can have any value from 0x00 to 0xFF. DO NOT try to simply assign such binary data to a string (in any programming language). The purpose of binary encodings, such as base64, hex, etc. are to encode binary data in printable us-ascii chars. Thus, if you the desired encrypted output is to be stored in a string, it must be encoded. Hex encoding uses 2 chars per byte (i.e. 0x01 because “01”). Base64 is a more compact string representation of binary bytes.