Proposed errata for DOM2 Range regarding insertNode()

[Ian Hickson]

DOM2 Traversal and Range has a number of problems, and really needs a
rewrite. However, in the absence of the resources to do that, I realised
that we could settle for releasing some errata. Arguably we as a working
group have somewhat the authority to do that, so here's a proposal for a
simple errata for DOM2 Range. If this works process out well, I'll see if
there are other things in DOM2 we should errata sooner rather than waiting
for wholesale DOM3 or DOM4 updates.

DOM2 Range says of the insertNode() method that "A node may be inserted
into a Range using the following method". However, if the range is
collapsed, according to a strict reading of the specification, calling
insertNode() actually inserts the node _after_ the range. I propose that
we change the spec to explicitly say that if you call insertNode() on a
collapsed range, the end point offset is increased by one, as if the node
was inserted before the end point, which I believe to be the intent of the
specification. This is implemented by Opera and WebKit already, and is
tested by Acid3.

If the working group chair will forgive me, I suggest we set a deadline of
May 21st (a week from today) at which point if there have been no
objections raised we go ahead and make the change to the DOM2 errata.

Cheers,
--
Ian Hickson U+1047E )\._.,--....,'``. fL
http://ln.hixie.ch/ U+263A /, _.. \ _\ ;`._ ,.
Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'

[Olli Pettay]

Ian Hickson wrote:
> DOM2 Range says of the insertNode() method that "A node may be inserted
> into a Range using the following method". However, if the range is
> collapsed, according to a strict reading of the specification, calling
> insertNode() actually inserts the node _after_ the range. I propose that
> we change the spec to explicitly say that if you call insertNode() on a
> collapsed range, the end point offset is increased by one, as if the node
> was inserted before the end point,
inserted node can be document fragment in which case end offset should
be increased more than one.


> which I believe to be the intent of the
> specification. This is implemented by Opera and WebKit already, and is
> tested by Acid3.
>
Gecko does what the current spec says.

> If the working group chair will forgive me, I suggest we set a deadline of
> May 21st (a week from today) at which point if there have been no
> objections raised we go ahead and make the change to the DOM2 errata.
Does web content rely on the current spec behavior?
Did Webkit or Opera change their behavior just to pass ACID3 (but not to
follow Range spec)?



-Olli

[Boris Zbarsky]

Ian Hickson wrote:
> I propose that we change the spec to explicitly say that if you call insertNode() on a
> collapsed range, the end point offset is increased by one

Increased at what point in time, exactly? Specifically, if there is a
DOMNodeInserted listener that repositions the range when the node insertion
happens, or even mutates the DOM, what is the expected behavior?

-Boris

[Ian Hickson]

On Wed, 14 May 2008, Olli Pettay wrote:
>
> > which I believe to be the intent of the specification. This is
> > implemented by Opera and WebKit already, and is tested by Acid3.
>
> Gecko does what the current spec says.

All three implementations do what the spec says now -- the spec can be
interpreted both ways.


> Does web content rely on the current spec behavior?

To my knowledge, very little Web content relies on this spec at all.
That's why Acid3 tested it, to make it interoperable enough that it could
be used. :-)


> Did Webkit or Opera change their behavior just to pass ACID3 (but not to
> follow Range spec)?

I didn't check whether they implemented this before Acid3 or not.


On Wed, 14 May 2008, Boris Zbarsky wrote:
> Ian Hickson wrote:
> > I propose that we change the spec to explicitly say that if you call
> > insertNode() on a collapsed range, the end point offset is increased by one
>
> Increased at what point in time, exactly? Specifically, if there is a
> DOMNodeInserted listener that repositions the range when the node
> insertion happens, or even mutates the DOM, what is the expected
> behavior?

DOM2 Range doesn't define anything to this level of detail yet,
unfortunately. For example regular old insertions and deletions near
ranges cause changes to the range values but the spec doesn't say if this
is before or after the events.

It seems that for sanity we should say it happens before, if we specify
this. Should we do this as a separate errata?

--
Ian Hickson U+1047E )\._.,--....,'``. fL
http://ln.hixie.ch/ U+263A /, _.. \ _\ ;`._ ,.
Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'

[Robert Sayre]

On Wed, May 14, 2008 at 6:15 PM, Ian Hickson <***@hixie.ch> wrote:
>
>
> To my knowledge, very little Web content relies on this spec at all.
> That's why Acid3 tested it, to make it interoperable enough that it could
> be used. :-)

I thought Acid3 tested things that have been written down for at least
5 years or something like that.

confused,

Rob

--

Robert Sayre

"I would have written a shorter letter, but I did not have the time."

[Robert Sayre]

On Wed, May 14, 2008 at 8:37 PM, Ian Hickson <***@hixie.ch> wrote:
> On Wed, 14 May 2008, Robert Sayre wrote:
>>
>> I'm not sure why you're sending private email about this. I asked a
>> simple question on the public list.
>
> I replied privately to the first question because it seemed rude to spam
> the list about it. I've cc'ed www-archive on this mail.

I've changed it back to public-webapi. If you ask me, it seems a bit
rude to get all control freak about the venue and then republish what
was private mail.

>
>
>> Olli wrote:
>> "Gecko does what the current spec says."
>>
>> So clearly there is some disconnect about the nature of the change
>> you're proposing.
>
> Sure. Like I said, the spec right now contradicts itself. If it didn't
> there wouldn't be much reason to ask for errata. :-)

Why not fix it and supercede it?


>
> Included for archival purposes:
>
>> On Wed, May 14, 2008 at 6:48 PM, Ian Hickson <***@hixie.ch> wrote:
>> > On Wed, 14 May 2008, Robert Sayre wrote:
>> >> On Wed, May 14, 2008 at 6:23 PM, Ian Hickson <***@hixie.ch> wrote:
>> >> > On Wed, 14 May 2008, Robert Sayre wrote:
>> >> >> On Wed, May 14, 2008 at 6:15 PM, Ian Hickson <***@hixie.ch> wrote:
>> >> >> >
>> >> >> > To my knowledge, very little Web content relies on this spec at
>> >> >> > all. That's why Acid3 tested it, to make it interoperable enough
>> >> >> > that it could be used. :-)
>> >> >>
>> >> >> I thought Acid3 tested things that have been written down for at
>> >> >> least 5 years or something like that.
>> >> >
>> >> > DOM2 Range came out in November 2000.
>> >>
>> >> So it tests what was written in 2000 or the proposed as yet unwritten
>> >> change?
>> >
>> > The spec right now does support what Acid3 does, it's just that it is
>> > ambiguous in its requirments and can be interpreted the other way around
>> > as well. Contradictions in specs happen, just like bugs in software.
>
> --
> Ian Hickson U+1047E )\._.,--....,'``. fL
> http://ln.hixie.ch/ U+263A /, _.. \ _\ ;`._ ,.
> Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
>



--

Robert Sayre

"I would have written a shorter letter, but I did not have the time."

[Ian Hickson]

On Wed, 14 May 2008, Robert Sayre wrote:
> >
> > Sure. Like I said, the spec right now contradicts itself. If it didn't
> > there wouldn't be much reason to ask for errata. :-)
>
> Why not fix it and supercede it?

As I said in the very first e-mail on this subject, that's what I'd like
to do. However, that's a significantly higher cost (years vs weeks) than
releasing an errata, and it was my impression that the Mozilla community
would like a quick turnaround on this.

What I recommend is that we release an errata as soon as possible, while
simultaneously looking for someone to rewrite the DOM Level 2 "Traversal
and Range" specification.

--
Ian Hickson U+1047E )\._.,--....,'``. fL
http://ln.hixie.ch/ U+263A /, _.. \ _\ ;`._ ,.
Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'

[Robert Sayre]

On Wed, May 14, 2008 at 9:02 PM, Ian Hickson <***@hixie.ch> wrote:
>
> As I said in the very first e-mail on this subject, that's what I'd like
> to do. However, that's a significantly higher cost (years vs weeks) than
> releasing an errata, and it was my impression that the Mozilla community
> would like a quick turnaround on this.

It looks to me like you're retroactively specifying something in your
test. I asked a simple question to determine whether that was the
case, since I would like to know the exact nature of the bugs Google
is (rather pompously) flaming us for.[1]

If there is disagreement about a change to normative behavior, it
seems like the right thing to do would be to discuss it, not pick one
interpretation and try to jam it through as errata. I don't see how
one can claim the spec can be interpreted both ways, but also that the
intent is clear. Is one of the possible interpretations a real
stretch?

[1] http://diveintomark.org/archives/2008/05/07/when-the-fall-is-all-thats-left

--

Robert Sayre

"I would have written a shorter letter, but I did not have the time."

[Ian Hickson]

On Wed, 14 May 2008, Robert Sayre wrote:
> On Wed, May 14, 2008 at 9:02 PM, Ian Hickson <***@hixie.ch> wrote:
> >
> > As I said in the very first e-mail on this subject, that's what I'd
> > like to do. However, that's a significantly higher cost (years vs
> > weeks) than releasing an errata, and it was my impression that the
> > Mozilla community would like a quick turnaround on this.
>
> It looks to me like you're retroactively specifying something in your
> test.

The test verifies that when you call insertNode() on a range, the node
that is passed is inserted into the range, as is required by DOM2 range
section 2.9. Inserting Content, sentences 1 and 2 (before and after the
code snippet).

These sentences are contradicted by the more generic sentences in section
2.12.1. Insertions, which don't take insertNode() into account for the
case of a collapsed range, and thus end up not implementing the
requirement in the former section.

I hold that the intent of the spec is clear, in that it would be pretty
dumb for an API for inserting nodes into a range didn't actually insert
nodes into a range; however, I agree that it is possible to interpret the
spec in a way that assums that the generic rules in the latter section
override the statements in the former section, hence my proposal that we
raise this as an errata.


> If there is disagreement about a change to normative behavior, it seems
> like the right thing to do would be to discuss it, not pick one
> interpretation and try to jam it through as errata.

Right, that's why I raised on the list, so that we can discuss it.

--
Ian Hickson U+1047E )\._.,--....,'``. fL
http://ln.hixie.ch/ U+263A /, _.. \ _\ ;`._ ,.
Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'

[Olli Pettay]

Ian Hickson wrote:
> The test verifies that when you call insertNode() on a range, the node
> that is passed is inserted into the range, as is required by DOM2 range
> section 2.9. Inserting Content, sentences 1 and 2 (before and after the
> code snippet).

The 2nd sentence doesn't require adding node to range. It talks about
context tree which isn't the same thing as range.

-Olli

[Olli Pettay]

Olli Pettay wrote:
> Ian Hickson wrote:
>> The test verifies that when you call insertNode() on a range, the node
>> that is passed is inserted into the range, as is required by DOM2 range
>> section 2.9. Inserting Content, sentences 1 and 2 (before and after the
>> code snippet).
>
> The 2nd sentence doesn't require adding node to range. It talks about
> context tree which isn't the same thing as range.
>
> -Olli
>

"A node may be inserted into a Range using the following method:"
- 'may' is pretty vague.

"The insertNode() method inserts the specified node into the Range's
context tree. "
- In 2.2.1 "... the content of a Range must be entirely within the
subtree rooted by a single Document, DocumentFragment or Attr Node.
This common ancestor container is known as the root container of the
Range. The tree rooted by the root container is known as the Range's
context tree."
So that doesn't say that the node is inserted to Range, but to
context tree.

"The node is inserted at the start boundary-point of the Range, without
modifying it."
What does the 'it' actually refer to? Boundary-point or Range or node?
And since end boundary-point isn't mentioned, one could argue that the
behavior defined in 2.12.1 should be used. When the range is collapsed,
the following:
"Note that when content is inserted at a boundary-point, it is
ambiguous as to where the boundary-point should be repositioned if
its relative position is to be maintained. There are two
possibilities: at the start or at the end of the newly inserted
content. We have chosen that in this case neither the container nor
offset of the boundary-point is changed. As a result, the
boundary-point will be positioned at the start of the newly inserted
content."



-Olli

[Olli Pettay]

Olli Pettay wrote:
>
> Olli Pettay wrote:
>> Ian Hickson wrote:
>>> The test verifies that when you call insertNode() on a range, the node
>>> that is passed is inserted into the range, as is required by DOM2 range
>>> section 2.9. Inserting Content, sentences 1 and 2 (before and after the
>>> code snippet).
>>
>> The 2nd sentence doesn't require adding node to range. It talks about
>> context tree which isn't the same thing as range.
>>
>> -Olli
>>
>
> "A node may be inserted into a Range using the following method:"
> - 'may' is pretty vague.
>
> "The insertNode() method inserts the specified node into the Range's
> context tree. "
> - In 2.2.1 "... the content of a Range must be entirely within the
> subtree rooted by a single Document, DocumentFragment or Attr Node.
> This common ancestor container is known as the root container of the
> Range. The tree rooted by the root container is known as the Range's
> context tree."
> So that doesn't say that the node is inserted to Range, but to
> context tree.
>
> "The node is inserted at the start boundary-point of the Range, without
> modifying it."
> What does the 'it' actually refer to? Boundary-point or Range or node?
> And since end boundary-point isn't mentioned, one could argue that the
> behavior defined in 2.12.1 should be used. When the range is collapsed,
> the following:
> "Note that when content is inserted at a boundary-point, it is
> ambiguous as to where the boundary-point should be repositioned if
> its relative position is to be maintained. There are two
> possibilities: at the start or at the end of the newly inserted
> content. We have chosen that in this case neither the container nor
> offset of the boundary-point is changed. As a result, the
> boundary-point will be positioned at the start of the newly inserted
> content."
>
>
>
> -Olli
>
>
>
So I'm not sure the errata for this issue is actually needed.
(In which case ACID3 should be changed)

[Ian Hickson]

Since the idea of making it clear that insertNode() inserts the node
inside the range even if the range is collapsed seems to have received a
somewhat positive response, I'd like to propose the following actual
errata text:

| DOM Level 2 Traversal and Range
|
| range-2. 2008-06-... [clarification]. Range.insertNode
|
| The sentence:
|
| The node is inserted at the start boundary-point of the Range, without
| modifying it.
|
| Should read:
|
| The node is inserted at the start boundary-point of the Range, without
| modifying the start boundary-point. If the range is collapsed, the
| offset of the Range's end boundary-point will be increased so that it
| is before the same node or character as it was before the insertion.

This uses the language style of the spec as it stands today (we can worry
about making the spec use RFC2119 terminology when we rewrite it later;
doing that now too would be a big rathole IMHO).

Chaals, can we put this to the working group as a proposed errata?


On Wed, 14 May 2008, Boris Zbarsky wrote:
> > >
> > > [discussion regarding mutation event timing]
> >
> > For example regular old insertions and deletions near ranges cause
> > changes to the range values but the spec doesn't say if this is before
> > or after the events.
>
> I think it's pretty clear that it should be when the actual mutation
> occurs (so before the events for insert cases and after for removal
> cases), but that does mean that one can't implement Range on top of DOM
> MutationEvents. Then again, there's no much one can implement on top of
> them, so that's OK, I think.
>
> > It seems that for sanity we should say it happens before, if we
> > specify this. Should we do this as a separate errata?
>
> I just wanted to point out that we have to be very careful in how we
> phrase this erratum (which I agree is a possibly useful one): we
> basically want to perform the insertion, then before firing the mutation
> event adjust the insertion endpoint after all the nodes we just
> inserted. Or something. In a UA that would fire multiple events here
> when inserting a DocumentFragment, we might lose no matter what we try
> to do.

I'm not sure what to do about mutation events. I think it may be wiser to
punt on this or at least deal with it in a separate errata.


On Fri, 23 May 2008, Olli Pettay wrote:
>
> So I'm not sure the errata for this issue is actually needed.

It seems to me that everyone agrees that insertNode() was always intended
to insert a node _into_ the range, and that the collapsed case was simply
lost between the cracks when the DOM WG was writing the spec (much as was
interaction with mutation events, for instance). While I agree that the
spec could be read as saying that the node is inserted after the range
when it is collapsed, I don't think we want that behaviour, nor that it
was intended. Do you actually think that behaviour is preferred to the
insertion behaviour?

--
Ian Hickson U+1047E )\._.,--....,'``. fL
http://ln.hixie.ch/ U+263A /, _.. \ _\ ;`._ ,.
Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'

[Jonas Sicking]

Ian Hickson wrote:
> On Fri, 23 May 2008, Olli Pettay wrote:
>> So I'm not sure the errata for this issue is actually needed.
>
> It seems to me that everyone agrees that insertNode() was always intended
> to insert a node _into_ the range, and that the collapsed case was simply
> lost between the cracks when the DOM WG was writing the spec (much as was
> interaction with mutation events, for instance).

Everyone who? And based on what? I don't see anything in the spec that
suggests that. And as Olli pointed out there is clearly language in the
spec that indicates that the inserted node would be after the range in
the collapsed case.

I guess I'm fine with making the change to the spec, but it would be a
change and not an errata. And if we're making changes anyway, I would
requests that we make NodeIterators behave like TreeWalkers as far as
the returnvalue for the NodeFilter goes. That is both more useful and
easier to implement since it allows more code reuse.

/ Jonas

[Olli Pettay]

Ian Hickson wrote:
> On Fri, 23 May 2008, Olli Pettay wrote:
>> So I'm not sure the errata for this issue is actually needed.
>
> It seems to me that everyone agrees that insertNode() was always intended
> to insert a node _into_ the range, and that the collapsed case was simply
> lost between the cracks when the DOM WG was writing the spec (much as was
> interaction with mutation events, for instance). While I agree that the
> spec could be read as saying that the node is inserted after the range
> when it is collapsed, I don't think we want that behaviour, nor that it
> was intended. Do you actually think that behaviour is preferred to the
> insertion behaviour?

The current version of the spec handles dom mutations consistently
(whether using node.insertBefore/appendChild or range.insertNode)
and that is not something I'd like to lose.

If that is changed, the insertNode part of the following wouldn't be
true anymore (2.12):
"Any mutation of the document tree which affect Ranges can be considered
to be a combination of basic deletion and insertion operations. In fact,
it can be convenient to think of those operations as being accomplished
using the deleteContents() and insertNode() Range methods and, in the
case of Text mutations, the splitText() and normalize() methods."


Let's have a testcase [1]:
<root><element1/><element2/></root>
and collapsed range is between element1 and element2.

element3 is inserted using the current version of insertNode():
<root><element1/><element3/><element2/></root>
Range stays collapsed and is pointed right after element1.

If element3 is inserted using root.insertBefore(element3, element2):
<root><element1/><element3/><element2/></root>
Range stays collapsed and is pointed right after element1, so the same
as with insertNode().

Now, if insertNode starts to work differently when range is collapsed
the result would be:
<root><element1/><element3/><element2/></root>, but range wouldn't be
collapsed anymore, instead start boundary point would be after element1
and end boundary point after element3. And this means that mutation
can't be thought "of ...as being accomplished using ... insertNode()
Range method..."
Or is there some combination of insertNode()s and deleteContents()s
which would lead to the collapsed range after <element1/>?


FF2, FF3, Opera9.2x and Opera9.5b have the, IMO, correct behavior,
Safari3.1 doesn't nor does ACID3'ed-Opera[2]


-Olli

[1] http://mozilla.pettay.fi/moztests/range_test.xhtml
[2] http://labs.opera.com/news/2008/03/28/

[Ian Hickson]

Chaals, please see the end of this message.

On Wed, 28 May 2008, Jonas Sicking wrote:
> >
> > It seems to me that everyone agrees that insertNode() was always
> > intended to insert a node _into_ the range, and that the collapsed
> > case was simply lost between the cracks when the DOM WG was writing
> > the spec (much as was interaction with mutation events, for instance).
>
> Everyone who? And based on what? I don't see anything in the spec that
> suggests that. And as Olli pointed out there is clearly language in the
> spec that indicates that the inserted node would be after the range in
> the collapsed case.

Well, everyone except you and Olli apparently. :-)

Do you really think that it was intended for insertNode() to act
differently when the range was collapsed than when the range wasn't
collapsed, with respect to whether the inserted node ends up in the range
or not?


> I guess I'm fine with making the change to the spec, but it would be a
> change and not an errata.

I'm not sure what the distinction is.


> And if we're making changes anyway, I would requests that we make
> NodeIterators behave like TreeWalkers as far as the returnvalue for the
> NodeFilter goes. That is both more useful and easier to implement since
> it allows more code reuse.

I recommend putting this forward as a separate errata. I only suggested
this one because implementations seem to be differing on how to implement
that part of the spec, and it seemed like it would be worth having a cycle
that didn't require finding an editor to resolve the problem.

On the long term I do think we need to get Traversal Range rewritten with
strict conformance requirements in mind.


On Thu, 29 May 2008, Olli Pettay wrote:
>
> The current version of the spec handles dom mutations consistently
> (whether using node.insertBefore/appendChild or range.insertNode) and
> that is not something I'd like to lose.

This seems like a consistency that would hurt the authors, though. What's
the use case for insertNode() inserting after the range?


> If that is changed, the insertNode part of the following wouldn't be
> true anymore (2.12):
> "Any mutation of the document tree which affect Ranges can be considered
> to be a combination of basic deletion and insertion operations. In fact,
> it can be convenient to think of those operations as being accomplished
> using the deleteContents() and insertNode() Range methods and, in the
> case of Text mutations, the splitText() and normalize() methods."

Indeed.


I do think that the spec intended the behaviour that I am proposing we
clarify in errata, though I agree that it could be read otherwise. I
honestly don't see any reason for the behaviour you advocate, it seems
confusing, not useful, and not in line with the intent of the spec.

However, I'm not sure how to make further progress if you disagree to this
change. It would be helpful if we could get a working group decision on
this so that I could know whether the Acid3 test should be changed.

--
Ian Hickson U+1047E )\._.,--....,'``. fL
http://ln.hixie.ch/ U+263A /, _.. \ _\ ;`._ ,.
Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'

[Jonas Sicking]

Ian Hickson wrote:
>
> Chaals, please see the end of this message.
>
> On Wed, 28 May 2008, Jonas Sicking wrote:
>>> It seems to me that everyone agrees that insertNode() was always
>>> intended to insert a node _into_ the range, and that the collapsed
>>> case was simply lost between the cracks when the DOM WG was writing
>>> the spec (much as was interaction with mutation events, for instance).
>> Everyone who? And based on what? I don't see anything in the spec that
>> suggests that. And as Olli pointed out there is clearly language in the
>> spec that indicates that the inserted node would be after the range in
>> the collapsed case.
>
> Well, everyone except you and Olli apparently. :-)

Who is "everyone"? I've only seen you stating that the spec can be
interpreted different ways after Olli pointed out what the definition of
"context tree" is.

> Do you really think that it was intended for insertNode() to act
> differently when the range was collapsed than when the range wasn't
> collapsed, with respect to whether the inserted node ends up in the range
> or not?

"Act differently" is a matter of how you view it.

>> I guess I'm fine with making the change to the spec, but it would be a
>> change and not an errata.
>
> I'm not sure what the distinction is.

W3C has in the past been very reluctant to making changes to specs, even
when specs have been incompatible with the web.

If we decide that it's ok to make functional changes to the spec we're
opening the flood gates for many more changes than merely clarifications.

/ Jonas

[Boris Zbarsky]

Ian Hickson wrote:
> DOM2 Range doesn't define anything to this level of detail yet,
> unfortunately.

Indeed. The wonders of Conway's Law...

Nevertheless, this question is somewhat important in terms of deciding where the
range should be positioned.

> For example regular old insertions and deletions near
> ranges cause changes to the range values but the spec doesn't say if this
> is before or after the events.

I think it's pretty clear that it should be when the actual mutation occurs (so
before the events for insert cases and after for removal cases), but that does
mean that one can't implement Range on top of DOM MutationEvents. Then again,
there's no much one can implement on top of them, so that's OK, I think.

> It seems that for sanity we should say it happens before, if we specify
> this. Should we do this as a separate errata?

I just wanted to point out that we have to be very careful in how we phrase this
erratum (which I agree is a possibly useful one): we basically want to perform
the insertion, then before firing the mutation event adjust the insertion
endpoint after all the nodes we just inserted. Or something. In a UA that
would fire multiple events here when inserting a DocumentFragment, we might lose
no matter what we try to do.

-Boris