Discussion:
[Interest] QCborValue and Qt types
Konstantin Shegunov
2018-12-01 11:54:35 UTC
Permalink
Hello,
I'm evaluating Qt's (5.12) cbor implementation for a network transmission
protocol. I saw that few Qt types were tagged already (I imagine as a proof
of concept/preview). Are there plans to extend these to include more types
(e.g. QImage) or should I just stick to writing them as binary buffers?
As a follow-up what is the suggested way to pass *custom* types to QCbor*,
are they to be streamed through QVariant?

Thanks in advance!
Kind regards.
Thiago Macieira
2018-12-01 17:21:49 UTC
Permalink
On Saturday, 1 December 2018 03:54:35 PST Konstantin Shegunov wrote:
> Hello,
> I'm evaluating Qt's (5.12) cbor implementation for a network transmission
> protocol. I saw that few Qt types were tagged already (I imagine as a proof
> of concept/preview). Are there plans to extend these to include more types
> (e.g. QImage) or should I just stick to writing them as binary buffers?
> As a follow-up what is the suggested way to pass *custom* types to QCbor*,
> are they to be streamed through QVariant?

Yes and no. See the attached draft spec (tag numbers are outdated) for a few
types I had been thinking of and already have an implementation for. Which
also reminds me I needed to send them to the CBOR IETF group for review.

I was thinking of sticking at first to QtCore types, so QImage will not be
included. I'm not even sure how we could add support for types in other
libraries; possibly you'll need to register a serialisation and
deserialisation function like qRegisterStreamOperators does.

And also note that we'll never get actual matching to Qt types. Like in the
attached geometry proposal, there's one type for size, not one for size and
one for size-float. This is important if you look at classes that are
identical but of different generations, like QVector4D and QQuaternion, or
worse functionally identical ones like QPixmap, QBitmap and QImage. We won't
get a standard tag that distinguishes them, since that distinction is not
useful outside of Qt. I'm open to ideas on how we can address that.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center
Konstantin Shegunov
2018-12-02 17:12:15 UTC
Permalink
Thanks for the heads up.

On Sat, Dec 1, 2018 at 7:43 PM Thiago Macieira <***@intel.com>
wrote:

> Yes and no. See the attached draft spec (tag numbers are outdated) for a
> few
> types I had been thinking of and already have an implementation for. Which
> also reminds me I needed to send them to the CBOR IETF group for review.
>

Looked them up, they seem fine to me. Hopefully the IETF agree.


> I was thinking of sticking at first to QtCore types, so QImage will not be
> included. I'm not even sure how we could add support for types in other
> libraries; possibly you'll need to register a serialisation and
> deserialisation function like qRegisterStreamOperators does.
>

Yes, this is how I imagined it initially. I suspect this is how it's done
currently for registered meta types?


> And also note that we'll never get actual matching to Qt types. Like in
> the
> attached geometry proposal, there's one type for size, not one for size
> and
> one for size-float. This is important if you look at classes that are
> identical but of different generations, like QVector4D and QQuaternion, or
> worse functionally identical ones like QPixmap, QBitmap and QImage. We
> won't
> get a standard tag that distinguishes them, since that distinction is not
> useful outside of Qt. I'm open to ideas on how we can address that.
>

Yes, I didn't think of that, but it makes sense. I suppose it should be
treated on a case-by-case basis. For the former I'd say something like
"geometry transformation" could be used, i.e. a catch-all for the
quaternions and/or matrices.
For the latter it's harder to judge, but probably it would be necessary to
use some kind of device-independent pixel format. If data encoding can be
used (e.g. png/jpeg) all the better, but even raw pixels should serve fine.
Granted this means that QPixmap and QBitmap may need to be converted to
QImage and vice versa, but I really see no clever way around it, at least
off the top of my head. That is unless it's acceptable to encode
meta-information like bit depth and such in the data stream, then maybe
some small optimization could be possible. Probably this'd mean that the
user will need to "know" beforehand what exact kind of object was put in
the cbor document, but without a schema I'd say that's probably to be
expected.
Thiago Macieira
2018-12-02 19:17:34 UTC
Permalink
On Sunday, 2 December 2018 09:12:15 PST Konstantin Shegunov wrote:
> Thanks for the heads up.
>
> On Sat, Dec 1, 2018 at 7:43 PM Thiago Macieira <***@intel.com>
> wrote:
> > Yes and no. See the attached draft spec (tag numbers are outdated) for a
> > few
> > types I had been thinking of and already have an implementation for. Which
> > also reminds me I needed to send them to the CBOR IETF group for review.
>
> Looked them up, they seem fine to me. Hopefully the IETF agree.
>
> > I was thinking of sticking at first to QtCore types, so QImage will not be
> > included. I'm not even sure how we could add support for types in other
> > libraries; possibly you'll need to register a serialisation and
> > deserialisation function like qRegisterStreamOperators does.
>
> Yes, this is how I imagined it initially. I suspect this is how it's done
> currently for registered meta types?

Again "yes and no". QtCore knows a little about the existence of QtGui and
QtWidgets, so qvariant.cpp and qmetatype.cpp have provisions to store the
hooks for the operations on types from those libraries. It's messy code and
has been a source of problems. For example, if v1 is a QtCore type and v2 is a
QtGui type, v1 == v2 might be true but v2 == v1 might not. Messy. Jędrzej
mentioned once he has a nice idea on how to fix it but couldn't fit it in the
margins of his diary.

But if we do support CBOR serialisation and deserialisation of types from
QtGui, internally QCborValue would have a couple of callbacks for QtGui to
post-process a decoded CBOR tag + payload.

Note also that the vast majority of standardiseable types from QtGui are also
types which we're discussing to move from QtGui to a non-GUI library.
Previously I objected to having them in QtCore with the rationale that "QtCore
is not the kitchen sink", but I'm open to new arguments.

That leaves the non-standardiseable ones, like QCursor, QTextFormat or QPen.
And yet we'd like to transmit them over CBOR. The only question is how. One
simple way is to reserve a metatype for "Qt 5 type" (or Qt 6), whose payload
is an array consisting of an integer (the metatype ID) and a binary payload.
That binary payload may be the same as QDataStream currently carries, or an
improvement thereof. But if we're going to do this, we will need to be better
at documenting the different payloads.

Or we just reserve a tag for "QDataStream", consisting of version number and
binary payload.

Or both.

> For the latter it's harder to judge, but probably it would be necessary to
> use some kind of device-independent pixel format. If data encoding can be
> used (e.g. png/jpeg) all the better, but even raw pixels should serve fine.
> Granted this means that QPixmap and QBitmap may need to be converted to
> QImage and vice versa, but I really see no clever way around it, at least
> off the top of my head. That is unless it's acceptable to encode
> meta-information like bit depth and such in the data stream, then maybe
> some small optimization could be possible. Probably this'd mean that the
> user will need to "know" beforehand what exact kind of object was put in
> the cbor document, but without a schema I'd say that's probably to be
> expected.

I think this is a good opportunity for us and the larger IoT community to
discuss how to best transport such data. Compressing as JPEG or PNG may be
good and well, but so is just plain transmitting of RGB/RGBA or YUV or YCbCr
data. Imagine a fingerprint scanner device that wants to transmit its reading
and let's say it's a 96x256 grayscale pixmap[1]. The device may not have
sufficient processing power to compress the image as PNG, but 24576 bytes may
be totally doable, even if it takes 26 UDP packets (960 payload bytes per
1024-byte datagram).

[1] I have no idea if this is a good representation nor am I saying anyone
should try this for their own devices without further investigation. This is
just for the sake of illustration.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center
Konstantin Shegunov
2018-12-05 23:14:16 UTC
Permalink
On Sun, Dec 2, 2018 at 9:45 PM Thiago Macieira <***@intel.com>
wrote:

> Again "yes and no". QtCore knows a little about the existence of QtGui and
> QtWidgets, so qvariant.cpp and qmetatype.cpp have provisions to store the
> hooks for the operations on types from those libraries. It's messy code
> and
> has been a source of problems. For example, if v1 is a QtCore type and v2
> is a
> QtGui type, v1 == v2 might be true but v2 == v1 might not. Messy. Jędrzej
> mentioned once he has a nice idea on how to fix it but couldn't fit it in
> the
> margins of his diary.
>

He did address the mailing list with a repository containing a
proof-of-concept of sorts for how the meta-type system may be
(re)implemented in the future. I admit I haven't given it a thorough
review, but it at a glance(!) looks okay.

But if we do support CBOR serialisation and deserialisation of types from
> QtGui, internally QCborValue would have a couple of callbacks for QtGui to
> post-process a decoded CBOR tag + payload.
>

That would probably be needed. Serializing only the types from QtCore would
constrain the usage of CBOR (in Qt) significantly, I think.

Note also that the vast majority of standardiseable types from QtGui are
> also
> types which we're discussing to move from QtGui to a non-GUI library.
> Previously I objected to having them in QtCore with the rationale that
> "QtCore
> is not the kitchen sink", but I'm open to new arguments.
>

I don't have anything of real substance to add to that. Perhaps only a
simple "outsider's view" - QtCore does seem to have gained quite a lot of
fat over the years, and moving yet more things into it will make it even
plumper, so I do understand your reluctance. But perhaps it's then prudent
to consider trimming it up by moving the non-core stuff into a separate
module which would be exactly the "kitchen sink", i.e. we need that around
but we don't need these classes very often. Unfortunately that'd require
some kind of "usage statistic" on what's used often and what not, which
isn't available to my knowledge. But even then if that's solved (in some
fashion) the question remains, what are the "core" classes, and what
classes are not, i.e. they're utilities and support that can be moved into
another binary. The distinction isn't always perfectly clear. Of course
adding yet another binary to deploy is also not to be taken lightly ... :|

That leaves the non-standardiseable ones, like QCursor, QTextFormat or
> QPen.
> And yet we'd like to transmit them over CBOR. The only question is how.
> One
> simple way is to reserve a metatype for "Qt 5 type" (or Qt 6), whose
> payload
> is an array consisting of an integer (the metatype ID) and a binary
> payload.
> That binary payload may be the same as QDataStream currently carries, or
> an
> improvement thereof. But if we're going to do this, we will need to be
> better
> at documenting the different payloads.


> Or we just reserve a tag for "QDataStream", consisting of version number
> and
> binary payload.
>

That'd be the "easy" way out, I guess. However it'd be really much better
if sensible data can be transmitted without resorting to QDataStream. You
yourself have said that QDataStream is portable and reliable as long as a
Qt application talks to a Qt application, but in some cases it _could_
(speculating here) be beneficial to have the data of a cursor (i.e a shape)
or QPen (i.e. width, color, cap, etc.) be available as separate entities to
a non-Qt application, and in a way that the hypothetical non-Qt application
can read it.

I think this is a good opportunity for us and the larger IoT community to
> discuss how to best transport such data. Compressing as JPEG or PNG may be
> good and well, but so is just plain transmitting of RGB/RGBA or YUV or
> YCbCr
> data. Imagine a fingerprint scanner device that wants to transmit its
> reading
> and let's say it's a 96x256 grayscale pixmap[1]. The device may not have
> sufficient processing power to compress the image as PNG, but 24576 bytes
> may
> be totally doable, even if it takes 26 UDP packets (960 payload bytes per
> 1024-byte datagram).
>

Yes, indeed this is what I had in mind. For a large image between desktop
applications it's definitely good to have some kind of compression
(preferably a good one) of the pixel data, but on the other hand that logic
falls apart if you're sending rather small datasets, or your device
couldn't handle the CPU requirements.

On Mon, Dec 3, 2018 at 4:59 PM Jason H <***@gmx.com> wrote:

> Kinda wondering why you just don't use QVariant and a QDataStream? You're
> just encoding as text, and if you need to use images, there's no current
> encoding (AFAIK) for pixel data in CBOR, so you can't interop with anything
> else anyway...
>
> I found CBOR limiting and not worth it. I was very interested in the
> promise/premise of CBOR, but I went QVariant+QDS and was glad I did.
>

Hi Jason,
Thanks for the input, I do appreciate it. I was just evaluating the
possibilities, more than committing to it. I have done my fair share of
network programming, and indeed QDS works just fine. It may be required to
have the server opened to non-Qt applications in the (more distant) future
however, so QDS fails in that regard. I can't make the clients match their
understanding to the QDS' versions and encodings, so that's why I was
inquiring of what's possible and what not.
Thiago Macieira
2018-12-06 17:12:18 UTC
Permalink
On Wednesday, 5 December 2018 15:14:16 PST Konstantin Shegunov wrote:
> Jędrzej
>
> > mentioned once he has a nice idea on how to fix it but couldn't fit it in
> > the
> > margins of his diary.
>
> He did address the mailing list with a repository containing a
> proof-of-concept of sorts for how the meta-type system may be
> (re)implemented in the future. I admit I haven't given it a thorough
> review, but it at a glance(!) looks okay.

Yeah, that was in response to my making that joke about the margins of his
diary :-)

Last time someone did that[1], the world was left with 400 years of mystery to
figure out what they meant.

[1] https://en.wikipedia.org/wiki/Fermat's_Last_Theorem

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center
Andrew Ialacci
2018-12-02 18:12:56 UTC
Permalink
Hope it’s ok that I barge in :P

Are there any recommended strategies for serializing custom model objects?

For example a Person class with name, age, and gender properties.

Or would I just write a

QByteArray toCBOR() const

method on all my model classes and create the CBOR object representation manually like with JSON.

Sent from my iPhone

> On Dec 1, 2018, at 6:43 PM, Thiago Macieira <***@intel.com> wrote:
>
>> On Saturday, 1 December 2018 03:54:35 PST Konstantin Shegunov wrote:
>> Hello,
>> I'm evaluating Qt's (5.12) cbor implementation for a network transmission
>> protocol. I saw that few Qt types were tagged already (I imagine as a proof
>> of concept/preview). Are there plans to extend these to include more types
>> (e.g. QImage) or should I just stick to writing them as binary buffers?
>> As a follow-up what is the suggested way to pass *custom* types to QCbor*,
>> are they to be streamed through QVariant?
>
> Yes and no. See the attached draft spec (tag numbers are outdated) for a few
> types I had been thinking of and already have an implementation for. Which
> also reminds me I needed to send them to the CBOR IETF group for review.
>
> I was thinking of sticking at first to QtCore types, so QImage will not be
> included. I'm not even sure how we could add support for types in other
> libraries; possibly you'll need to register a serialisation and
> deserialisation function like qRegisterStreamOperators does.
>
> And also note that we'll never get actual matching to Qt types. Like in the
> attached geometry proposal, there's one type for size, not one for size and
> one for size-float. This is important if you look at classes that are
> identical but of different generations, like QVector4D and QQuaternion, or
> worse functionally identical ones like QPixmap, QBitmap and QImage. We won't
> get a standard tag that distinguishes them, since that distinction is not
> useful outside of Qt. I'm open to ideas on how we can address that.
>
> --
> Thiago Macieira - thiago.macieira (AT) intel.com
> Software Architect - Intel Open Source Technology Center
> <cbor-tags-geometry.rst>
> <cbor-tag-bitarray.rst>
> _______________________________________________
> Interest mailing list
> ***@lists.qt-project.org
> https://lists.qt-project.org/listinfo/interest
Thiago Macieira
2018-12-02 19:20:13 UTC
Permalink
On Sunday, 2 December 2018 10:12:56 PST Andrew Ialacci wrote:
> Hope it’s ok that I barge in :P
>
> Are there any recommended strategies for serializing custom model objects?
>
> For example a Person class with name, age, and gender properties.
>
> Or would I just write a
>
> QByteArray toCBOR() const
>
> method on all my model classes and create the CBOR object representation
> manually like with JSON.

I'm not sure you meant to return a binary to be placed inside a CBOR payload
or if that is the CBOR serialisation. Either way, I don't think returning
QByteArray is a good idea. Instead, return QCborValue, such that the value may
be placed inside a larger object or array if the caller so wishes.

Or if you want to do streaming, just void serialize(QCborStreamWriter &);

Similar for decoding.
--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center
Andrew Ialacci
2018-12-03 13:13:49 UTC
Permalink
You are exactly correct. Returning a QCborValue is what I want to do. Was writing on phone in a movie theater. :P

One other question:

For the core parts of our infrastructure we use RapidJSON rather than QTs JSON classes as it’s over twice as performant.

Do you know off hand how how QTs CBOR implantation would stack up against RapidJSON in terms of pure object serialization performance?

Thank you sincerely for your time 🍍

Sent from my iPad

> On Dec 2, 2018, at 8:33 PM, Thiago Macieira <***@intel.com> wrote:
>
>> On Sunday, 2 December 2018 10:12:56 PST Andrew Ialacci wrote:
>> Hope it’s ok that I barge in :P
>>
>> Are there any recommended strategies for serializing custom model objects?
>>
>> For example a Person class with name, age, and gender properties.
>>
>> Or would I just write a
>>
>> QByteArray toCBOR() const
>>
>> method on all my model classes and create the CBOR object representation
>> manually like with JSON.
>
> I'm not sure you meant to return a binary to be placed inside a CBOR payload
> or if that is the CBOR serialisation. Either way, I don't think returning
> QByteArray is a good idea. Instead, return QCborValue, such that the value may
> be placed inside a larger object or array if the caller so wishes.
>
> Or if you want to do streaming, just void serialize(QCborStreamWriter &);
>
> Similar for decoding.
> --
> Thiago Macieira - thiago.macieira (AT) intel.com
> Software Architect - Intel Open Source Technology Center
>
>
>
> _______________________________________________
> Interest mailing list
> ***@lists.qt-project.org
> https://lists.qt-project.org/listinfo/interest
Thiago Macieira
2018-12-03 16:52:39 UTC
Permalink
On Monday, 3 December 2018 05:13:49 PST Andrew Ialacci wrote:
> Do you know off hand how how QTs CBOR implantation would stack up against
> RapidJSON in terms of pure object serialization performance?

You're not comparing apples to apples.

And I have not yet optimised the serialisation code, only the deserialisation
one. The serialisation is still using QBuffer or QIODevice in all cases.

--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center
Jason H
2018-12-03 14:58:59 UTC
Permalink
_______________________________________________
Interest mailing list
***@lists.qt-project.org
https://lists.qt-project.org/listinfo/interest
Loading...