Etsi, XAdES-BES, XAdES-EPES, FacturaE, Electronic Invoicing, etc.

Chilkat v9.5.0.75 will include features to make it easy to generate XAdES-BES and XAdES-EPES signatures for electronic invoicing and tax reporting for many countries.

The goal is to provide an easy and inexpensive solution for these complicated requirements.  Chilkat has been working with customers in Spain, Italy, Hungary, India, Brazil, Costa Rica, and elsewhere to get things working smoothly.  (Contact Matt at support@chilkatsoft.com for more information.)

The vision is to offer an online tool that generates the solution (source code) given a sample of already-signed XML.  This tool is live and functioning at https://tools.chilkat.io/xmlDsigGen.cshtml. The tool can generate code in any of approximately 30 different programming languages.  The tool will be updated and refined as Chilkat solves issues and challenges brought by the requirements of new customers.   XML digital signatures can be tricky, fragile, brittle, and different implementations for checking the validity of a signature can be quirky, have undocumented requirements, etc.  The idea is to build into the code generation tool the expertise for knowing the particular needs of different authorities, so you don’t have to spend weeks of frustration in getting things to work.

As time passes, and as more expertise is embodied within the code generation tool, I believe Chilkat can reach the point where it “just works”.

 

How to Encrypt with no Padding (encrypted output size equals input size)

Block encryption algorithms, such as AES, will produce output that is a multiple of the algorithm’s block size. For AES, the output is a multiple of 16 bytes. However, this is for the typically used cipher modes “CBC” (Cipher Block Chaining) and “ECB” (Electronic Cookbook).

There are other cipher modes that turn a block algorithm into a stream cipher. See https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation

Namely, the cipher modes are:

  • Counter Mode (ctr)
  • Cipher Feedback (cfb)
  • Output Feedback (ofb)

In C++, for example, any of the above modes can be used with AES to produce output that is exactly the same size as the input.
For example


    CkCrypt2 crypt;

    crypt.put_CryptAlgorithm("aes");
    // Use Counter mode.  Other stream modes are "cfb" and "ofb".
    crypt.put_CipherMode("ctr");
    crypt.put_KeyLength(128);
    crypt.SetEncodedKey("000102030405060708090A0B0C0D0E0F","hex");
    crypt.SetEncodedIV("A0A1A2A3A4A5A6A7A8A9AAABACADAEAF","hex");

    const unsigned char plainText[] = { 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23 };

    CkByteData data;
    data.append2(plainText,23);

    CkByteData encData;
    crypt.EncryptBytes(data,encData);

    printf("size of encrypted data = %d\n",encData.getSize());
    const unsigned char *pEncData = encData.getData();

IMPORTANT: The output of encryption consists of bytes that resemble random data, where each byte can have any value from 0x00 to 0xFF. DO NOT try to simply assign such binary data to a string (in any programming language). The purpose of binary encodings, such as base64, hex, etc. are to encode binary data in printable us-ascii chars. Thus, if you the desired encrypted output is to be stored in a string, it must be encoded. Hex encoding uses 2 chars per byte (i.e. 0x01 because “01”). Base64 is a more compact string representation of binary bytes.

HMAC Hex Key Ambiguity

This happens all the time..

Quite often, a service provider will provide instructions for HMAC generation, and will provide a hexadecimal HMAC key in the example, such as:

01A0251D601FEF3305A38B124068A001CF3F099AA187DB2886314C67CDFDEBCD

This is implicitly ambiguous because there are two ways to interpret the instructions:

1) The HMAC key is composed of the ascii bytes ‘0’, ‘1’, ‘A’, ‘0’, ‘2’, ‘5’, etc. In other words, the HMAC key is 0x30, 0x31, 0x41, etc.

or

2) The HMAC key is composed of the bytes represented by the hex string. In other words, the HMAC key is 0x01, 0xA0, 0x25, etc.

Both can be accomplished using Chilkat.
For the #1 case, one would call

crypt.SetMacKeyString("01A0251D … 67CDFDEBCD")

For the #2 case, one would call

crypt.SetMacKeyEncoded("01A0251D … 67CDFDEBCD","hex")

POP3 Error: No X-UIDL header found

The Chilkat MailMan class can fetch emails from a POP3 server in two ways: by sequence number, or by UIDL. When an email is fetched by UIDL, or fetched in a way such that a full mapping of UIDL’s to sequence numbers was retrieved, Chilkat will add an “X-UIDL” header to the Email object that is returned. This allows for the email object to be uniquely associated with the email on the server. (Sequence numbers change with each POP3 session, but UIDL’s don’t.)

For convenience, an email object is passed to some MailMan methods, such as w/ DeleteEmail. If the email was not retrieved in a way such that the UIDL was known, then the X-UIDL header will not be present, and the MailMan has no way of specifying which email on the server to delete. For example, if the email was downloaded by calling MailMan.FetchByMsgnum, then no UIDL was ever known (unless perhaps MailMan.GetUidls was previously called in the same POP3 session). Thus the error message in the LastErrorText is “No X-UIDL header found”.

One solution is to call MailMan.GetUidls beforehand. One call at the start of the POP3 session is sufficient.

SSH/SFTP Error: Must first connect to the SSH server

The following error is explained in this post:

ChilkatLog:
  DownloadFileByName:
    DllDate: Apr 25 2018
    ChilkatVersion: 9.5.0.73
    UnlockPrefix: *
    Architecture: Little Endian; 64-bit
    Language: Cocoa Objective-C
    VerboseLogging: 0
    SftpVersion: 3
    Component successfully unlocked using purchased unlock code.
    Must first connect to the SSH server.
  --DownloadFileByName
--ChilkatLog

The above error can happen after a long period of inactivity. Let’s say your application successfully connected and authenticated w/ the SFTP server, did some things, and then did not do anything else for a long period of time.  Meanwhile, the SFTP server decides to disconnect because the client has been inactive for too long.  The client (your app + Chilkat) would only discover that the server has dropped the connection once it tries to do something, such as in a call to DownloadFileByName.  The non-connected socket is discovered in the 1st attempt to send a message, and thus you receive the above error.

There are two possible actions an application might take:

  1. Prevent the disconnect by periodically calling sftp.SendIgnore to keep the connection from being inactive.
  2. (auto-recovery)  If DownloadFileByName (or some other method) returns false/0 to indicate failure, examine the sftp.IsConnected property.  If not connected, then automatically re-connect, re-authenticate, and call InitializeSftp to get back to a connected state, and then retry the method.

Chilkat v9.5.0.73 Release Notes

The v9.5.0.72 release notes are available here: Chilkat v9.5.0.72 Release Notes

v9.5.0.73 Release Notes:

  • Email The SetHtmlBody method, in certain circumstances, would incorrectly set the top-level MIME header to text/html for multipart messages. This was fixed.
  • HTTP Non us-ascii chars in the URL path are now always URL encoded using the utf-8 encoding.
  • OAuth2 Added the UseBasicAuth and AppCallbackUrl properties.
  • MailMan Added methods SendMimeBd and FetchMimeBd.
  • Bounce Fixed a few situations where bounce type 11 (Suspected Bounce) was returned, but should have been 6 (Auto-Reply). In general, minor improvements to Bounce categorization are added as customers report issues, and each new Chilkat version can be assumed to have minor Bounce categorization refinements.
  • Tar Fixed crash (access violation) in the Untar method. This was caused by a corrupt tar file where the internal header contained garbage bytes.
  • Objective-C/CkoJsonObject In iPhoneOS11.2.sdk/usr/include/complex.h, we find the following macro: “#define I _Complex_I”. This caused a compile error because the CkoJsonObject class has a property named “I”. Chilkat updated CkoJsonObject.h to “#undef I” to avoid the conflict.
  • Electron Added builds for Electron 1.8. However the package naming triggered some npm spam detectors (false positives of course), and Chilkat is working to get it resolved. Chilkat will produce builds for Electron 2.0 in the near future.
  • CkByteData Added a SecureClear bool property (get_SecureClear, put_SecureClear). If set to true, then whenever the internal data is deallocated, the memory is first overwritten with 0 bytes.
  • MIME Binary MIME with null bytes in some bodies became corrupted because 0 bytes were replaced with SPACE chars. This is fixed.
  • Signed/Encrypted Email Fixed: The “micalg” attribute incorrectly remained in the Content-Type header field for the encrypted MIME part when the email is also signed.
  • REST Fixed: Stream sources used for uploading were not properly closed after the upload finished.
  • C++ Builder/CkAuthAzureStorage Fixed: The x-ms-date header did not automatically get the correct current date/time for HTTP requests. This only happened for C++ Builder and Delphi builds of Chilkat.
  • Email Added flexibility in parsing non-compliant RFC822 date strings, where the month name and month day number are not in the correct order as specified by RFC822.
  • MIME parsing (general) Added more internal flexibility for handling mixtures of CRLF and bare-LF line endings.
  • XmlDSigGen Now capable of using non-exportable private keys on Windows, such as for A3 certificates where the private key is on a hardware token.
  • SSH Fixed a rarely encountered “handshake” problem.
  • HTTP Changed the default Content-Type for the PostJson method to be “application/json”. The original default value, “application/jsonrequest” was the initial “standard” years ago, but seems to never be used nowadays. If an “application/jsonrequest” is needed (and I doubt it will ever be needed), then PostJson2 may be called to explicitly specify the Content-Type.
  • HTTP The default value of the S3Ssl property is now true.
  • Zip Fixed rare problems involved with rewriting zip archives, when the “move from temp zip to target” fails.
  • Email Fixed certain automatic MIME structuring issues w.r.t. multipart/alternative and multipart/related.
  • Compression Added the DeflateLevel property to the Compression class.
  • CkString The removeDelimited method was missing for Ruby, Java, Perl, Python, Tcl, and PHP.
  • SFTP Fixed: The SyncTreeDownload method was not firing the DownloadRate callback.
  • PrivateKey Fixed: The GetPkcs8Pem method (for ECC keys) was returning PKCS1 but should’ve been returning PKCS8.
  • HTTP Added the SharePointOnlineAuth method.
  • HTTP Fixed problems with the S3_GenerateUrl and S3_GenerateUrlV4 methods.
  • SSH Fixed: After doing a ReadDir, the SFtpFile.IsDirectory property was not correct for some types of SSH servers.
  • Zip Added the PwdProtCharset property.
  • SOCKS5 Fixed problems with IPv6 addresses when using SOCKS5 proxies.
  • Compression Added the CompressSb and DecompressSb methods.
  • JSON Added the methods DtOf and DateOf to both JsonObject and JsonArray.
  • HTTP Fixed problems with non-us-ascii chars in URLs for downloads.
  • StringBuilder Added the ReplaceAfterFinal method.
  • HTTP Fixed: The LastHeader property was empty after the PostUrlEncoded method.
  • SSH Fixed slowness for SSH commands the emit a large amount of output.
  • HTTP Fixed rare server certificate verification problem when the server provides out-of-order certificates in the TLS handshake.
  • FTP2 Fixed: The IdleTimeoutMs property was not being honored for DNS problems.
  • StringBuilder Added the WriteFileIfModified method.
  • PureBasic Fixed a compile error in the CkHttp.pb file.
  • SSH/SFTP/SshTunnel Fixed a host key signature verification failure for certain situations.
  • SSH/SFTP/SshTunnel Added the UncommonOptions property, which will be a place to specify future unforeseen workarounds that may be required for particular SSH servers (old or new).
  • Rest/Socket Fixed a problem when a non-standard HTTP port (not 80 nor 443) is used in for the original Socket connection in conjunction with the Rest.UseConnection method.
  • RSA Added the SetX509Cert method to make it easy to use the private key of certificate. Also allows for A3 certificates where the private key is non-exportable (on a Windows system) such as on a hardware token.
  • XmlDSigGen Added the “X509Data+KeyValue” option for the KeyInfoType property.
  • JsonArray Added the FindString and FindObject methods.
  • Zip Fixed reliability issues in the UnzipToStream method.
  • XmlDSigGen Fixed: The X509SerialNumber in the X509IssuerSerial needed to be in decimal, not hex.
  • S/MIME Fixed an extremely rare issue where a digital signature verification failed but should’ve been successful.
  • JsonObject Fixed the “Unable to lock my JSON object.” error that would be returned if SetStringOf was called on an empty JsonObject.
  • CkString The loadFile method will now recognize Unicode/utf-8 BOMs and will load files correctly based on the BOM encountered.

Windows 10 1803 can’t run EXE files from a network shared folders

Chilkat has been receiving support email with the following error:

          connectSocket:
            connect_ipv6_or_ipv4:
              resolveHostname6:
                getAddressInfo:
                  Failed to get host address info. (3)
                  SocketError: Error 0x2afb
                  Check to make sure the connection is not blocked by a firewall or anti-virus port filtering.
                  hostOrIpAddr: ****
                  port: 
                  See http://www.chilkatforum.com/questions/11627/sftp-failed-to-get-address-info for a possible cause of this error.
                  Versions of Windows earlier than Windows XP are limited to handling IPv4 only
                  On Windows Server 2003 and Windows XP, IPv6 addresses are returned only if IPv6 is installed on the local computer.
                --getAddressInfo
              --resolveHostname6
              Domain to IP address resolution failed.
            --connect_ipv6_or_ipv4
          --connectSocket

One user described the situation perfectly:

This issue I am having is with FTP2 and the Email components. I have enclosed the error screen for the FTP.

  1. I did a windows update last night. It has worked fine for 2 years until the update.
  2. There is no issue if I run the program in the VB6 interpreter. It only happens when I run the .exe program.
  3. If I run the .exe program from the local drive, it works fine. It only happens when I run it from the network drive.
  4. I have another computer set up and it works fine from the network drive.

Please let me know if you have an idea. I am running windows 10 (32bit)


The answer is found here.

In summary: “It can be concluded that Windows 10 update 1803 for security reasons does not allow you to open network connections in programs running from shared folders that are accessible only using the SMBv1 protocol. As network folders, you need to use devices that support SMBv2 or SMBv3.”

See http://woshub.com/windows-10-1803-cant-run-exe-files-shared-folders/ for more details.

Yield and SleepMs in PowerBuilder

This is a note for PowerBuilder programmers: Some Chilkat classes provide a SleepMs method, which is provided as a convenience. The SleepMs method puts the thread to sleep for a number of milliseconds. However, this is not the same as a PowerBuilder Yield, which “Yields control to other graphic objects, including objects that are not PowerBuilder objects. Yield checks the message queue and if there are messages in the queue, it pulls them from the queue.”

The above was found by a Chilkat customer w/ regard to the Task.SleepMs method:

The problem was the when we were calling awParent.Dynamic wf_download_percent(percent) to display the download percentage on a progress window of ours, the window wasn’t always getting control to show that! I needed to add a Yield() right after that and then it was perfect. That’s a fairly common problem in PowerBuilder. I have to admit I would have thought that your SleepMs method would be tantamount to a Yield(), but obviously it’s not.

Perhaps you might consider adding something to your PowerBuilder samples that show progress of Async actions, where instead of just logging progress it was calling a made-up function to display progress, and then add a Yield() after that to make it clear that is needed.

Chilkat v9.5.0.72 Release Notes

The v9.5.0.71 release notes are available here: Chilkat v9.5.0.71 Release Notes

v9.5.0.72 Release Notes:

  • FTP2 Fixed FTP implicit SSL/TLS uploads for some FTP servers.
  • CkString Fixed: In the Delphi DLL build, the CkString.split* and tokenize* methods caused a crash.
  • Encryption Modification: Encrypting 0 bytes for block cipher algorithms with padding now results in one block of output, rather than 0 bytes.
    (Decrypting the 16-byte block will return the original 0 bytes.) This only applies to block ciphers operating in modes such as ECB, CBC, etc. This does not apply to stream ciphers, or block ciphers operating in a streaming mode.
  • XML Fixed cases where “ ” was getting changed to 
 in round-trip load/save.
  • HTTP The Http.AwsSignatureVersion property default value is changed from 2 to 4. All AWS regions support v4 signatures, whereas S3 regions deployed after January, 2014 do not support V2.
  • PrivateKey Fixed a problem in loading certain (seldom encountered) types of private keys. Specifically, keys using PBES1/RC4.
  • PKCS7 Signature Fixed certain cases where creating PKCS7 signatures embedded the same certificate twice.