v9.5.0.63 Release Notes

Version 9.5.0.63 is a new version that almost immediately follows v9.5.0.62.   It will only be released for a few programming languages (ActiveX, .NET, Delphi, and Excel).   Chilkat will provide this release in other programming languages / operating system upon request.

The reason for releasing is that Chilkat wishes to create Quickbooks REST API examples, and it was discovered that the Quickbooks server-side is unable to consume HTTP requests that use MIME header folding.  Therefore, Chilkat added an AllowHeaderFolding property to both the Http and Rest classes.

There were a few other very minor fixes / additions:

  • Added the Email.AddAttachmentBd method.
  • Added the Http.DownloadBd and http.DownloadSb methods.  (See the previous release notes for information about “Bd” and “Sb” methdos: http://cknotes.com/v9-5-0-62-release-notes/ )
  • Made the Http.RequireSslCertVerify method more flexible in handling out-of-order certs from a web server.  See http://www.chilkatforum.com/questions/11402/certificate-order-issue-enhancement

v9.5.0.62 Release Notes

  • Added StringTable class.  The StringTable class differs from the StringArray class in that it’s intended to hold a table of strings that is mostly read-only.  The internal storage of all the strings is in a single contiguous block of memory for efficient (minimized) memory storage.
  • Expanded the BinData class by adding new methods and making it available in all programming languages.
  • Expanded the StringBuilder class by adding many more methods.
  • Note: BinData and StringBuilder are consistently the same across all programming languages, whereas CkString and CkByteData are not.   CkString and CkByteData will continue to be used in future versions, are still an integral part of Chilkat, and are NOT deprecated.
  • Note: One use of BinData and StringBuilder is in use of new methods ending in “Sb” and “Bd”.  These methods are for returning or passing text or binary data, especially large amounts of text or data.  For example, imagine one Chilkat method is used to retrieve a large amount of data, and then that data is passed to another Chilkat method.  A large waste of time and memory is consumed in returning the string (or binary bytes) to the programming language, and then immediately passing it back to Chilkat.  If it is .NET, then the data must be marshaled from unmanaged to managed, and then, to pass back to Chilkat it is marshaled from managed to unmanaged.  The same applies when using the ActiveX (with BSTR’s and Variants).  By retrieving results directly into a BinData or StringBuilder, and then passing the data by passing the object, the data stays on the native side.  For example, the new Rest.ReadRespSb method reads the REST response directly in to a StringBuilder.  If it is JSON, this can be loaded directly into a JsonObject via the JsonObject.LoadSb.  In the future, new “Sb” and “Bd” methods will be added in places where large amounts of data are likely to be encountered.  This will create pathways allowing data to stay native as much as possible.

    Version 9.5.0.62 adds the following initial set of Sb/Bd methods:

    JsonObject.LoadSb
    JsonObject.EmitSb
    Xml.LoadSb
    Xml.GetXmlSb

    Email.GetMimeBd
    Email.GetMimeSb
    Email.SetFromMimeSb
    Email.SetFromMimeBd

    Mime.LoadMimeSb
    Mime.LoadMimeBd
    Mime.GetMimeSb
    Mime.GetMimeBd
    MailMan.RenderToMimeBd
    MailMan.RenderToMimeSb
    Rest.FullRequestSb
    Rest.ReadRespSb
    Rest.SendReqSb
    Rest.SetMultipartBodySb
    Rest.AddQueryParam
    Rest.ReadRespBd
    Rest.SendReqBd
    Rest.SetMultipartBodyBd
    Imap.FetchAttachmentSb
    Imap.FetchAttachmentBd
    Imap.AppendMimeWithFlagsSb
    SFtp.UploadSb
    SFtp.DownloadSb
    SFtp.UploadBd
    SFtp.DownloadBd
    Ftp2.GetFileSb
    Ftp2.GetFileBd

    Ftp2.PutFileSb
    Ftp2.PutFileBd

  • SshTunnel – Now automatically keeps the SSH connection alive during idle times by sending IGNORE messages every 20 seconds (when idle).
  • The 32-bit .NET VS2015 build now uses the /LARGEADDRESSAWARE linker option.  (See /LARGEADDRESSAWARE)
  • Minor CSV fix where the last line in a CSV file was inadvertently being trimmed of whitespace on both ends.
  • Added the Global.UnlockStatus property to allow applications to programmatically know if the UnlockBundle (or UnlockComponent) method call succeeded because of the 30-day trial or a recognized/purchased unlock code.
  • Rest – allows Basic authentication for non-secure connections to localhost.  Normally, Chilkat disallows the Basic authentication method if the connection is not TLS.  There is one exception now, and that is for connections to localhost.

v9.5.0.61 Update: New Features, Fixes, Changes, etc.

  • REST: The FullRequestFormUrlEncoded method did not work correctly with AWS authentication.  This was fixed.  (Note: This fix is required for using Amazon SES to send email.)
  • TLS on Big Endian Machines: Fixed TLS connection issue with some cipher suites on Big Endian machines.   There was a particular internal piece of code that did not properly account for the endianness of the machine and assume little endian (which is by far the most common byte ordering).
  • FTP2: The “m”, “d”, and “y” XML attributes in the XML result returned by GetXmlDirListing were mixed up.  (For example: full=”20160929-100512″ m=”2016″ d=”9″ y=”29″ hh=”10″ mm=”5″ ss=”12″)   This was fixed.
  • SFTP: The ReadFileBytes method could hang for duration of IdleTimeoutMs if file size was smaller than number of bytes requested.  This was fixed.
  • SshTunnel: Added the IsSshConnected method.
  • SshTunnel: Added internal functionality to periodically send IGNORE messages to the SSH server to keep the connection alive when the clients are inactive.
  • BinData: The BinData object was previously specific to the ActiveX.  It is now available across all programming languages.  The BinData object duplicates some of the functionality of CkByteData, but CkByteData is specific to some programming languages.

v9.5.0.60 Update: New Features, Fixes, Changes, etc.

  • ActiveX, .NET: Fixed problems with the LastMethodSuccess property. Version 9.5.0.60 is a patch for .NET and ActiveX only for this one issue. The Mono builds, .NET Core, and .NET UWP (Windows Runtime) did not have any problem with LastMethodSuccess, and therefore remain at v9.5.0.59. Other languages (C++, Java, Perl, Python, etc.) are also unaffected and therefore remain at v9.5.0.59.

v9.5.0.59 Update: New Features, Fixes, Changes, etc.

  • OAuth2: Added the OAuth2 class.
  • FTP2: Fixed SyncRemoteTree/SyncRemoteTree2 progress monitoring callbacks.
  • Crypt2: Fixed the GenerateUuid to respect RFC 4122. The GenerateUuid method generates the version 4 variant, which is composed of entirely random data except for the few bits that signify version 4.
  • Compression: Fixed the CompressFile method for large files. See Fixed CompressFile
  • MHT: Fixed: MS-Word html files with %20 in file paths needed to be replaced with SPACE chars.
  • Cert/SMIME/PKCS7: Fixed: When a cert doesn’t have the Issuer CN, the encrypted message could not be decrypted using Chilkat library, even though the cert does have subject key id.
  • REST: Dropbox workaround: DropBox is not Q/B decoding header fields as it should do (if it were
    behaving according to standards). Two modifications were made: (1) Disable Q/B encoding for header fields when Dropbox is the endpoint. (2) Added a new REST property named AllowHeaderQB to enable/disable Q/B encoding for header fields. It is enabled by default.
  • SSH: Fixed: ChannelReceiveUntilMatch did not work for some cases with wildcard (*) match patterns.
  • Android/Java: Fixed the emoji utf-8 problem. The use of emoji’s caused issues with the underlying JNI implementation because Java uses modified-utf8, which is only different because it must use surrogate pairs for chars that encode to longer than 3 bytes. Chilkat will now (internally) emit the correct modified utf-8 using surrogate pairs to keep the JNI fat and happy (given that this problem was discovered w/ the pizza slice emoji for a major pizza chain).
  • HTTP/TrustedRoots: Fixed: When verifying the web server certificate for a TLS connection, the Windows registry-based root CA cert stores were not being searched when TrustSystemCaRoots was true.
    Now Chilkat will correctly search for the root cert, and if found, will trust the root cert.
  • PEM: Added the “AddItem” method. See the very bottom of this example: AddItem
  • MHT: Enhancement: MHT will not try to embed images when the data is already embedded, such as:
    <img style=”display:block;margin:0 auto 5px;text-align:center;” src=”data:image/png;base64,iVBORw0KGg…
  • HTTP: Fixed auto-follow of HTTP redirects when the redirect URL is a Protocol-relative_URL (i.e. URLs beginning with “//”)
  • Zip: Added the AppendDataEncoded method.
  • Python/FreeBSD: Fixed the installer for Python on FreeBSD.
  • Email: Fixed Q/B decoding issue for address related headers. The SPACE char between the friendly name and the email address was dropped when Q/B decoding.
  • JSON: Fixed a parse problem that occurred when a Unicode char was the very last char in a JSON value string. For example if the value string ended with “\\u6e2c”, then this final char was dropped in the string returned by various methods.
  • TLS 1.2: Fixed: The server would close the TLS connection during initial handshake (after receiving the ClientHello) if it picked the cipher suite TLS_ECDHE_ECDSA in TLS 1.2. The reason is that Chilkat did not include the ECDSA/SHA1 signature hash algorithm as a candidate for the Server Key Exchange. (Note: This was not a security issue — it was just a case where the TLS handshake would fail if the server picked a particular cipher suite, only in TLS 1.2, and expected to use a paricular Server Key Exchange signature hash algorithm.)
  • SSH: Added the hmac-sha1-96 algorithm to what is supported in the internal SSH connection protocol.
  • Email: Eliminated problems arising from X.400 email addresses where there is no “@” character.
  • HTTP: Fixed the ReceiveRate callback (so that it’s actually called) for the WinRT and Unicode C++ builds.
  • Unicode C++: Event callbacks were broken starting in v9.5.0.58 for the Unicode C++ builds. (They were working in v9.5.0.57, broken in .58, and now fixed in .59)
  • IMAP: A particular kind of email on a particular IMAP server would cause a hang (or timeout) when downloading the email for the case where AutoDownloadAttachments is disabled.
  • SFTP: Worked around problems with the Devart SSH server.
  • Email: Fixed email structure problems for certain situations when SetHtmlBody was called.
  • HTTP/S3: Fixed a problem caused when filenames contain SPACE chars.
  • Email: The ComputeGlobalKey method was deprecated and should not be used. The inputs to the MD5 hash algorithm were not properly canonicalized, and therefore the computed key could change from version to version (based on fixes, improvements, etc. in the MIME processing).
  • Email: Added the ComputeGlobalKey2 method, which is properly canonicalized to avoid the issues of the original ComputeGlobalKey implementation.
  • XMP: Fixed: TIFF files with EXIF IFD’s were not correctly re-written.
  • Email/MIME: Fixed problems where the values in address related header fields were not returned Q/B decoded. (For example, the value returned by the GetHeaderField method should be Q/B decoded.)
  • SSH: Fixed a performance problem w/ the ChannelPoll method for very small timeouts.
  • Zip: Fixed a problem w/ encrypted zips for Unicode C++ (CkZipW) on Mac OS X. The issue has to do with the fact that wchar_t is 4 bytes per char (on Mac OS X) as opposed to 2 bytes per char.
  • Email: Fixed GetSignedByCert so that it will still return the certificate even if the signature validation fails.
  • FTPS: Fixed a PBSZ/”PROT P” problem with the Global eXchange Services Secure FTP brand of FTP server.
  • JSON: Fixed a JSON parsing problem having to do with escaped chars.
  • MHT: Fixed a problem with “link” tags where attributes such as “media” were getting dropped.

OAuth2 for Desktop Applications (C#, VB.NET using Chilkat)

Here are C# and VB.NET sample applications (on GitHub) for doing OAuth2 authorization using Chilkat for Facebook, Google, LinkedIn, Salesforce, and GitHub.

OAuth2 Sample for Desktop Applications (C#) for Facebook, Google, LinkedIn, Salesforce, and GitHub

OAuth2 Sample for Desktop Applications (VB.NET) for Facebook, Google, LinkedIn, Salesforce, and GitHub

 

 

Damn You Windows Firewall!

One particular cause of the following error is finally known:

sockRecv(47ms):
    WindowsError: An existing connection was forcibly closed by the remote host.
    WindowsErrorCode: 0x2746
    numBytesRequested: 5
    Failed to receive data on the TCP socket
--sockRecv
sockRecv failed.
sockRecvN_buf: Did not receive the exact number of bytes desired.
numBytesToReceive: 5

This error was caused by the Windows Firewall doing stateful FTP filtering. The solution is to disable stateful FTP filtering so that the firewall does not block any FTP traffic. (This is typically prescribed for the server-side of FTP, but it also applies to the client-side.)

Note: The Windows Firewall’s stateful FTP filtering only causes problems when the connection is SSL/TLS. Clear (non-secured) connections are not affected.

To disable stateful FTP filtering, open an administrative command prompt, and type the following:

netsh advfirewall set global StatefulFTP disable

Also Note: The above error could also be a result of the connection truly being closed by the remote host, or something on the server-side. This solution only applies to the case where Windows Firewall (on the client-side computer) is interfering with a SSL/TLS FTP connection.

How to Reproduce the Problem

To reproduce the problem, write a program that establishes a TLS connection with an FTP server.   After authenticating, write a simple loop that calls ftp.Noop 1500 times.   If Windows Firewall is interfering, it is likely to fail in the same place every time.  For example, in my case the error occurs on iteration 699 every time.

Also, the error may not be easily reproducible even if Windows Firewall is a problem.  For example, the problem seemed to be not reproducible if the FTP server was not a Windows-hosted FTP server.   One might suspect the server-side to be the problem, but the connection reset ([RST, ACK] in a WireShark trace) definitely originated from the client-side.

Disabling the firewall’s Stateful Packet Inspection (SPI) feature solved the problem every time. (It should be the case that stateful packet inspection should NEVER try to inspect the packets of a TLS encrypted channel for the simple fact that it’s impossible to inspect.  The firewall does not have the ability to decrypt the packets in the channel, and therefore it cannot inspect anything. The firewall’s only choices SHOULD be to simply allow or disallow the connection.  I don’t understand why stateful packet inspection should ever be “monkeying around” with encrypted channels..)