Moho crash when trying to place a layer out of a group by "PlaceLayerBehindAnother()"

[Rai López]

Hi, posting again here before reporting as a bug just in case... I'm having serious problems when trying to recolocate layers in palette by means of "PlaceLayerBehindAnother" function. I've used it before under different scenarios without problems, so I suspect the problem here is try to use it over layers being dragged in layers palette. Plus, in other similar tests the lua console window has thrown the following kind of unexpected error message...



...just before crashing (I say unexpected because I've not made use of anything similar to "LayerByDepth" function in my scripts at all, so I guess it's an internal error that for some reason is leaking into the lua console window? I don't really know, but informing about it just in case is useful.

The part of the embedded script I'm trying to develop is a little more complex, it consists on not allowing you to drag a layer (normally the layer containing the script) out of the context it has been designed to act, but I have reduced it to the minimum to reproduce the crash as follows:

Code: Select all

function LayerScript(moho)
	doc = moho.document
	fixedLayer = moho.layer
	if oldLayerPos ~= nil then
		if oldLayerPos ~= doc:LayerAbsoluteID(fixedLayer) then
			print("Old Placement:   " .. oldLayerPos)
			print("New Placement:   " .. doc:LayerAbsoluteID(fixedLayer))

			moho:PlaceLayerBehindAnother(fixedLayer, doc:LayerByAbsoluteID(oldLayerPos))

			print("Final Placement: " .. oldLayerPos)
			print("------------------")
		end
	end
	oldLayerPos = doc:LayerAbsoluteID(fixedLayer)
end

It simply moves the layer you drag in palette to its original position (well, it fails if you drag it down at the same level, but that would require some more code beside the point), and here is the same script with a quick-testing project:

rl_fixed_layer_20220731-2258.zip

Where as soon as you drag the red layer into the group, the program crashes... Well, I've tried a lot of things, even the "PlaceLayerInGroup" one with similar results. I've also tried to ensure the correct layer IDs are provided at any time, but it doesn't seem to be the cause of the crashes as far as I can tell. The problem, as I said, seems to be more related to this functions somehow messing the way Moho deals with layers depth/ordering system and the fact a script try to move a layer into/out of a group at the same time it's being dragged by user in palette.

Well, I've tried to keep it simple, sorry if it still sounded messy after all... but maybe some of you can see something around a possible way to avoid the crashes or an explanation of why they happen I'm simply not able to. Thanks for reading in any case!

[Rai López]

BTW, I have been thinking lately if coding it in a way that it is based on layer's UUID instead of layer's AbsoluteID wouldn't be the key? But the fact there is not a direct way to access a layer by its UUID just like you can simply do with "LayerByAbsoluteID" puts me a little off of trying to do it... But if someone thinks it could be the way to avoid the crashes, now that well could worth the try.

The reason for this making some kind of sense to me is UUIDs never change no matter what (as far as I know), meantime AbsoluteIDs change accordingly layers order in palette in frame 0, and something tells me this could be what somehow interferes at some point with some Moho's internal task, probably related to depth sorting, and messes everything up.

Of course, for the layers UUID solution I should take into account layers above and/or bellow the affected layer instead of the UUID of the layer itself to make it work (or I think so), a little more complex to do too but, again, if it could work that way it well could worth the effort...

[synthsin75]

I don't think PlaceLayerBehindAnother is designed to work as a more general PlaceLayerInGroup.
And the LayerByDepth - out of range is likely because LayerByAbsoluteID was built on LayerByDepth...and you have to increment all the layer IDs above a moved layer to account for the shift.

It does sound like UUID would be a better solution. Just have to iterate through the document ( https://mohoscripting.com/snippets/1 ) and make a table of the UUIDs. Probably more overhead, but likely a necessary evil.

[SimplSam]

Whilst the issue is definitely there - I don't think there is a bug in the command - per se. As an example - it is used to move layers out of groups in the A.Evseeva script 'Place layer in/out of group': https://mohoscripts.com/script/ae_place_layer_in_group.

I think the issue is with the command being used with groups, when run inside a layer script. I suspect some reference is not updated somewhere and then Moho gets its knickers in a twist following the execution of the layer script.

[Rai López]

Thank you for answering, guys! You made me think I could indeed asking to much to "PlaceLayerBehindAnother" function by making use of it for all the moving tasks, so now I'm using "PlaceLayerInGroup" when a layer is dragged into a group (logic ) and the advised way in mohoscripting.com to move a layer out of its group (moho:PlaceLayerBehindAnother(moho.layer, moho.layer:Parent()) when necessary and... well, at least the crashes seem to be gone! There is still an "attempt to call a userdata value" lua console error at some point, but it's something

The modified code (sorry it's still a little rough) is as follows:

Code: Select all

function LayerScript(moho)
	local doc = moho.document
	local fixedLayer = moho.layer
	local fixedLayerAbsID = doc:LayerAbsoluteID(fixedLayer)

	if fixedLayerOldPos ~= nil then
		if fixedLayerOldPos ~= fixedLayerAbsID then
			local fixedLayerNewAbsID = doc:LayerAbsoluteID(fixedLayer)
			print("1. Old Placement:   " .. fixedLayerOldPos)
			print("2. New Placement:   " .. fixedLayerNewAbsID)

			if doc:LayerID(fixedLayer) < 0 and doc:LayerByAbsoluteID(fixedLayerNewAbsID):Parent() ~= nil and doc:LayerByAbsoluteID(fixedLayerNewAbsID):Parent():IsGroupType() then
				local newLayerPosParent = moho:LayerAsGroup(doc:LayerByAbsoluteID(fixedLayerNewAbsID):Parent())
				moho:PlaceLayerInGroup(doc:LayerByAbsoluteID(fixedLayerNewAbsID), newLayerPosParent, true, false)
			elseif doc:LayerByAbsoluteID(fixedLayerOldPos) and doc:LayerByAbsoluteID(fixedLayerOldPos):Parent() ~= nil then
				moho:PlaceLayerBehindAnother(fixedLayer, doc:LayerByAbsoluteID(fixedLayerOldPos):Parent())
			else
				moho:PlaceLayerBehindAnother(fixedLayer, doc:LayerByAbsoluteID(fixedLayerOldPos))
				moho:PlaceLayerBehindAnother(doc:LayerByAbsoluteID(fixedLayerOldPos), fixedLayer)
			end

			print("3. Final Placement: " .. doc:LayerAbsoluteID(fixedLayer))
		end
	end
	fixedLayerOldPos = doc:LayerAbsoluteID(fixedLayer)
end

It more or less works as expected (although the real use will defer depending on different situations) and if you try it out in the same tester file you'll see it shows funny behaviors in the lua console window, like the repeating printed step 3 just after the "attempt to call a userdata value" error when dragging the red layer into the group starting the drag from an inferior position, it's like what you said about Moho recalculating all the above new IDs or something... but it's kind of funny how it only repeats that part of the script.

Well, now I only have to figure out how to get rid of this new error for start considering it as workable/reliable, although I'm afraid at some point I could have to switch to the UUID method, yes... which sounds like more trusty when it turns to get rid of this kind of "attempt to call..." errors, but I'd like to continue the IDs path while possibilities last because I wanted to keep it as simple as possible and avoid another loop (if possible) plus have to rethink the way the fixed layer positions are obtained.

Well, thank you again! It's a pain in the neck when you clash with this kind of unexpected obstacles, but at the same time at least one (almost) always learn something

[synthsin75]

Sometimes Moho doesn't seem to like long commands, like: doc:LayerByAbsoluteID(fixedLayerNewAbsID):Parent():IsGroupType()
Seems to work better using one or two variables.

That's the only thing that stands out to me as a possible source for the "attempt to call a userdata value".

[Rai López]

Sometimes Moho doesn't seem to like long commands, like: doc:LayerByAbsoluteID(fixedLayerNewAbsID):Parent():IsGroupType()

Really? Never thought about that... But I wanted to clean such parts fruit of quick coding as one of my first next steps anyway, so I'll soon see if it makes a difference. Thanks for the advice!

[Rai López]

OK, another finding... the long commands doesn't seem to make a difference regarding to the "attempt to call a userdata value" error, BUT what do seem to make a difference is passing false to the third argument of "PlaceLayerInGroup" function so it places the layer at bottom of the group, in this case it will kept like this:

Code: Select all

moho:PlaceLayerInGroup(doc:LayerByAbsoluteID(fixedLayerNewAbsID), fixedLayerNewPosParent, false, false)

As soon as this is set to false instead true, not only the error disappears, but also the weird loop that made all that repeated prints of step 3. Now I have to test if I can reorder the moved layer at will afterwards without problems/surprises (but I don't see why not, since once the layer is in the group that shouldn't be a problem) and, although there are still weird behaviors around all this I can't explain and may be problematic at some point, in this case there could be a happy ending after all...

So the last code I'm trying that doesn't seem to present any problem no matter how and where I drop the tester project's red layer would be this:

Code: Select all

function LayerScript(moho)
	local doc = moho.document
	local fixedLayer = moho.layer
	local fixedLayerID = doc:LayerID(fixedLayer)
	local fixedLayerAbsID = doc:LayerAbsoluteID(fixedLayer)

	if fixedLayerOldPos ~= nil then
		if fixedLayerOldPos ~= fixedLayerAbsID then
			print("1. Old Placement:   " .. fixedLayerOldPos)
			local fixedLayerNewAbsID = doc:LayerAbsoluteID(fixedLayer)
			print("2. New Placement:   " .. fixedLayerNewAbsID)
			local fixedLayerNewPosParent = doc:LayerByAbsoluteID(fixedLayerNewAbsID):Parent()

			if doc:LayerID(fixedLayer) < 0 and fixedLayerNewPosParent ~= nil and fixedLayerNewPosParent:IsGroupType() then
				moho:PlaceLayerInGroup(doc:LayerByAbsoluteID(fixedLayerNewAbsID), fixedLayerNewPosParent, false, false)
			elseif doc:LayerByAbsoluteID(fixedLayerOldPos) and doc:LayerByAbsoluteID(fixedLayerOldPos):Parent() ~= nil then
				moho:PlaceLayerBehindAnother(fixedLayer, doc:LayerByAbsoluteID(fixedLayerOldPos):Parent())
			else
				moho:PlaceLayerBehindAnother(fixedLayer, doc:LayerByAbsoluteID(fixedLayerOldPos))
				moho:PlaceLayerBehindAnother(doc:LayerByAbsoluteID(fixedLayerOldPos), fixedLayer)
			end

			print("3. Final Placement: " .. doc:LayerAbsoluteID(fixedLayer))
		end
	end
	fixedLayerOldPos = doc:LayerAbsoluteID(fixedLayer)
end

EDIT: Yeah, confirmed I can send dropped layer to the top of the group (or wherever, really) after placing it at bottom for the exposed reasons with "PlaceLayerInGroup", so adding for example:

Code: Select all

moho:PlaceLayerBehindAnother(fixedLayer, fixedLayerNewPosParent:Layer(fixedLayerNewPosParent:CountLayers() - 1))
moho:PlaceLayerBehindAnother(fixedLayerNewPosParent:Layer(fixedLayerNewPosParent:CountLayers() - 1), fixedLayer)

It seems like a decent workaround for now... Now I still have to adapt all this to the real code and see, but I'm kind of seeing the light after several days of waste... I mean LEARNING! time

[Rai López]

OK, turned out that while trying to put the last findings into real code, I started to get crashes again, so I think I'll finally have to abandone the idea of get this working for now (I made some tries of making it work with UUIDs instead IDs with the same luck). It was a not essential feature for what I have in mind anyway, so it could have been worse... BUT, what I do think I have found is a way of easily demonstrate that there is a bug indeed around here, which is something at least. The thing is under same circumstances "PlaceLayerBehindAnother()" seem to work reliably with any kind of group type of layer (Bone, Particle, etc.) but NOT with "Group Layers", as can be quickly tested with this:

https://www.dropbox.com/s/uvnmnd4i8zhfk ... 4.zip?dl=1

And here is for the record the testing code, which is quite simple:

Code: Select all

function LayerScript(moho)
	local doc = moho.document
	local scriptLayer = moho.layer
	local scriptLayerID = doc:LayerID(scriptLayer)
	local scriptLayerAbsID = doc:LayerAbsoluteID(scriptLayer)

	if scriptLayerOldLoc ~= nil then
		if scriptLayerOldLoc ~= scriptLayerAbsID then
			local scriptLayerNewAbsID = doc:LayerAbsoluteID(scriptLayer)

			if doc:LayerID(scriptLayer) < 0 and doc:LayerID(doc:LayerByAbsoluteID(scriptLayerOldLoc)) then
				print(">> Dragged from root (" .. scriptLayerOldLoc .. "), dropped into group (" .. scriptLayerNewAbsID .. ") & returned to its original location (" .. scriptLayerOldLoc ..")")

				moho:PlaceLayerBehindAnother(scriptLayer, doc:LayerByAbsoluteID(scriptLayerOldLoc))
				moho:PlaceLayerBehindAnother(doc:LayerByAbsoluteID(scriptLayerOldLoc), scriptLayer)
			end

			print("1. Old Placement:   " .. scriptLayerOldLoc)
			print("2. Drop Placement: " .. scriptLayerNewAbsID)
			print("3. Final Placement:  " .. doc:LayerAbsoluteID(scriptLayer))
		end
	end
	scriptLayerOldLoc = doc:LayerAbsoluteID(scriptLayer)
end

As you can see if you try, you can drag & drop the red DragMeLayer into any of the group type layers and the layer returns to its original root location as expected and without problems, but as soon as you try to do the same with the Group Layer (Layer 3) the program simply crashes, so something weird is happening only in this case and, hopefully, that should easily point something for devs.

Oh, curiously, if you drop the DragMeLayer just over the Group Layer (and not inside between its children, for example), the crash doesn't seem to occur, not sure why it could make a difference, but there it is just in case it helps somehow.

Well, now I think I can make a more direct/decent report at least. Of course if some of you could lead into something for avoiding the crashes that would be very welcomed, but I'm afraid that it's time to wait for a fix... Thanks to everyone!

[synthsin75]

That's odd. If you set that group layer to not show (by ANY means), the script doesn't crash. You can turn the visibility off, make the opacity zero, hide in editing view, or hide in the layers window, and no crash. You can even animate visibility, and it will crash when visible on the timeline and not when invisible.


There's definitely some kind of UI-related bug going on here.

[Rai López]

That's odd. If you set that group layer to not show (by ANY means), the script doesn't crash..

That makes me think... What if, as a temporal workaround, I turned off visibility of the group (in case it is a Group Layer) just before calling PlaceLayerBehindAnother() and then make it visible again? Not sure if it will make a difference by doing it this way and from the script, but I'll definitelly try it out as soon as I can and see... Thanks for pointing that out!

[Rai López]

Well, I've tried everything that came to my mind related to disabling Group Layer's visibility but nothing seems to make a difference regarding to crashes, but still it was worth the try just in case... Here is what I've tried just in case someone realized I could be missing something else:

Code: Select all

function LayerScript(moho)
	local doc = moho.document
	local scriptLayer = moho.layer
	local scriptLayerID = doc:LayerID(scriptLayer)
	local scriptLayerAbsID = doc:LayerAbsoluteID(scriptLayer)

	if scriptLayerOldLoc ~= nil then
		if scriptLayerOldLoc ~= scriptLayerAbsID then
			local scriptLayerNewAbsID = doc:LayerAbsoluteID(scriptLayer)
			local scriptLayerNewParent = scriptLayer:Parent()

			if doc:LayerID(scriptLayer) < 0 and doc:LayerID(doc:LayerByAbsoluteID(scriptLayerOldLoc)) then
				print(">> Dragged from root (" .. scriptLayerOldLoc .. "), dropped into group (" .. scriptLayerNewAbsID .. ") & returned to its original location (" .. scriptLayerOldLoc ..")")

				if scriptLayerNewParent and scriptLayerNewParent:LayerType() == MOHO.LT_GROUP then --disabling all kind of Group Layer's visibility settings in order to see if it makes a difference for avoiding crashes (no luck)
					scriptLayerNewParent = moho:LayerAsGroup(scriptLayerNewParent)

					---[[trying different ways to force some kind of "update" BEFORE applying changes (no luck)
					--scriptLayerNewParent:UpdateCurFrame(true)
					--MOHO.Redraw()
					--moho:UpdateUI()
					--scriptLayerNewParent:Expand(not scriptLayerNewParent:IsExpanded())
					--]]

					scriptLayerNewParent:SetVisible(false)
					scriptLayerNewParent:SetRenderOnly(true)
					scriptLayerNewParent.fVisibility.value = false
					scriptLayerNewParent.fAlpha.value = 0

					---[[trying different ways to force some kind of "update" AFTER applying changes (no luck either)
					--scriptLayerNewParent:UpdateCurFrame(true)
					--MOHO.Redraw()
					--moho:UpdateUI()
					--scriptLayerNewParent:Expand(not scriptLayerNewParent:IsExpanded())
					--]]
				end

				moho:PlaceLayerBehindAnother(scriptLayer, doc:LayerByAbsoluteID(scriptLayerOldLoc))
				moho:PlaceLayerBehindAnother(doc:LayerByAbsoluteID(scriptLayerOldLoc), scriptLayer)
				
				if scriptLayerNewParent and scriptLayerNewParent:LayerType() == MOHO.LT_GROUP then --restoring Group Layer's visibility settings...
					scriptLayerNewParent:SetVisible(true)
					scriptLayerNewParent:SetRenderOnly(false)
					scriptLayerNewParent.fVisibility.value = true
					scriptLayerNewParent.fAlpha.value = 1
				end
			end

			print("1. Old Placement:   " .. scriptLayerOldLoc)
			print("2. Drop Placement: " .. scriptLayerNewAbsID)
			print("3. Final Placement:  " .. doc:LayerAbsoluteID(scriptLayer))
		end
	end
	scriptLayerOldLoc = doc:LayerAbsoluteID(scriptLayer)
end

As you can see, I've been also playing with different ways to force some kind of "update" before and/or after applying visibility changes, but it doesn't improved things in any way, quite the contrary as one could expect.

Well, having tried that, I think my very last chance is going to be to try to ensure the dropped layer is previously positioned at the very top of the Group Layer (I already tried that at some point with success, but not having all the knowledge about the crashes I've now), which seems to be the only way Moho don't crash and that's also the reason it works when layer is dropped just over the Group Layer itself instead inside and between/bellow it's children (as I pointed as a curiosity a pair of replies above). If that turns out to work reliably, it may be a way to make it work without complicating this not-essential part of the script too much and therefore it may still worth to invest a little more time on it... --> OK, after testing this more thoroughly, I don't think it could work, since I Moho seems to crash as soon as the script try to take out the dropped layer UNLESS it has been initially dropped at the very top position, anything other than that ends up crashing the program... So yeah, end of the road I'm afraid?

EDIT: Sorry I had to make some little corrections to the code above, just in case someone already tried it!

[synthsin75]

That's odd. If you set that group layer to not show (by ANY means), the script doesn't crash..

That makes me think... What if, as a temporal workaround, I turned off visibility of the group (in case it is a Group Layer) just before calling PlaceLayerBehindAnother() and then make it visible again? Not sure if it will make a difference by doing it this way and from the script, but I'll definitelly try it out as soon as I can and see... Thanks for pointing that out!

Yeah, that's what I tried too. The only other thing I can think of, that I didn't try, was maybe adding a LM.Snooze(msec) to see if Moho just needs a little time to update the UI.
But since it's only for group layers, I doubt that would work either.

[Rai López]

Yeah, that's what I tried too.

It simply sounded too good

The only other thing I can think of, that I didn't try, was maybe adding a LM.Snooze(msec) to see if Moho just needs a little time to update the UI.
But since it's only for group layers, I doubt that would work either.

Hmm... I don't think that would work because, as far as I know, the time while Moho is Snoozed no other change would be made at any level of the program, nevertheless I tried at some point to do something like that but by os.clock() run time comparisons, of course without success... but I think I failed because by its nature this script only runs ONCE (because IDs only can be different for the very fist run of the script). BUT, all we know embedded scripts are run several times no matter what, so the last thing I'm trying is force it to run two times at least and it DO seem to be making a difference! Although I don't what to say it too loud yet... But it makes sense, the idea is in the first run the dropped layer is just settled inside the Group Layer not doing anything else; AND, in the second run, PlaceLayerBehindAnother() is called ONLY to take the dropped and already settled layer out of the Group (the problematic part), but the key is doing it in the subsequent cycle, when the Group Layer seems to have ended to do whatever it does for making Moho crash if it's all done at the same time/cycle.

I hope it makes sense that way written... Anyway I'm going to try it out a little more and I'll post what I mean in code, if no surprises arise ()

[Rai López]

OK, not sure if the more elegant way to do it, but... it seems to WORK!

The code would be something like this (although I expect to somehow improve the "runTimes" logic implementation):

Code: Select all

function LayerScript(moho)
	local doc = moho.document
	local scriptLayer = moho.layer
	local scriptLayerID = doc:LayerID(scriptLayer)
	local scriptLayerAbsID = doc:LayerAbsoluteID(scriptLayer)
	runTimes = runTimes or 0

	if scriptLayerOldLoc ~= nil and runTimes ~= nil then
		if scriptLayerOldLoc ~= scriptLayerAbsID or runTimes < 1 then
			local scriptLayerNewAbsID = doc:LayerAbsoluteID(scriptLayer)

			if doc:LayerID(scriptLayer) < 0 and doc:LayerID(doc:LayerByAbsoluteID(scriptLayerOldLoc)) then
				print(">> Dragged from root (" .. scriptLayerOldLoc .. "), dropped into group (" .. scriptLayerNewAbsID .. ") & returned to its original location (" .. scriptLayerOldLoc ..")")

				if runTimes == 0 then
					moho:PlaceLayerBehindAnother(scriptLayer, doc:LayerByAbsoluteID(scriptLayerOldLoc))
					moho:PlaceLayerBehindAnother(doc:LayerByAbsoluteID(scriptLayerOldLoc), scriptLayer)
				end
			end

			print("1. Old Placement:   " .. scriptLayerOldLoc)
			print("2. Drop Placement: " .. scriptLayerNewAbsID)
			print("3. Final Placement:  " .. doc:LayerAbsoluteID(scriptLayer))

			runTimes = runTimes + 1
		end
	end

	if runTimes > 1 then print("runTimes > 1")
		runTimes = 0
	else
		scriptLayerOldLoc = doc:LayerAbsoluteID(scriptLayer)
	end
end

I'm using a slightly different Moho project that the one I posted before, but it could be equally tested on that as well, the thing is now you can drag & drop the red DragMeLayer wherever you want inside the Group Layer and it returns to its original position without any crashes 100% of the times (for now) here! Of course the best outcome is it gets fixed at some point (it is already reported), but at least meantime it seems I'm going to be able to use these layer placing functions more or less as I had in mind and without too much hassle or script over-complexity after all... Plus, with this solution I think everything should continue working if/after a fix finally arrives and, as I said, I've learned some cool things on the way, so: