And this gets to the crux of things. Let me see if I have this right... a named pred (let's say Crisis, or worse, Katrika, who is known to wait for and snack on strays) eats a stray, and since no one knows about her, no one goes looking for this missing person. Or by the time they do, the land vanishes back "home". Or it's a victim no one cares about. Or the victim's neighbors are left clueless. Or the city lacks the resources to chase said predator down. Or... or... or... it doesn't matter if a named pred only stumbles upon a random vanishing land and picks off clueless strays once in a lifetime or 10 times a year; somehow their luck never, ever runs out
Short of a military base getting caught up in a vanishing land, civilians from another world are about as effective against giant predators as a native Felaryan living in the middle of the jungle. Hiding is your best bet in those cases, get someplace small and tight, where the pred can't get at you, or even find you. The best thing people stuck on a vanishing land can do is hole up someplace they can secure. Most sentient preds are not going to walk up onto a vanishing land and risk getting taken away at random (As mentioned, even Crisis limits herself to picking off people who stray away from the VL). The only preds who would take that kind of risk are the very greedy, the risk-takers, or someone who just is completely ignorant to what a vanishing land is.
However, giant wildlife, such as Kensha Beasts, Tonorions, and the other many types of large, dangerous animals found in Felarya... they actually are a very real threat, since they are, well, animals and will go to wherever food is.
As far as people lusting for revenge... that is an overused cliche, and not necessarily how people think in a disaster situation. When you're in that kind of trouble, you worry about survival. If Steve gets killed, well that sucks for him. You can mourn later, but for now you have to survive. Any people that would run off into the jungle for "revenge" against a creature that can only be harmed by military-grade weapons are either incredibly stupid, or acting so irrational that, honestly, they deserve what they get. There would be people like that, but anyone who's lived in Felarya for any length of time would not be that stupid. Death is a part of life there, and if someone you know dies, you move on. You do what you have to do to survive.
This ain't like bombing an ocean to stop the tidal waves; sentient or not, there are killers out there to be targeted and dealt with. For a city that suddenly finds itself in Felarya this is an issue; and for the government that rules that city, Felarya, or more specifically, the predators within, becomes an issue.
All-out offense is a terrible idea against Felarya. It's an alien landscape full of a 1000 different ways to die between the poisonous plants, insects, reptiles, carnivorous plants, large wildlife (tiger-sized, bear-sized and so on), giant wildlife (kenshas, tonorions and so on) and finally, if you can even find one, the giant hybrids who are not only larger than the largest creature to ever exist on Earth, but also have intelligence, enhanced senses and magic.
Playing offense is how people get killed. Defense is the way to go. Stay hidden, dig out some places to hide, or if you have the resources, get your position fortified as best you can.
Fortunately, for Felarya, it's quite difficult to find that realm again after it has temporarily snatched one of your cities and one or more of its sentient/non-sentient denizens have taken the lives of a few of your people before you get that city back
My question is, why would you even try to go back? Why would your people even support that?
It isn't like the Felaryan creatures came to their world, and started wrecking things. The people on the vanishing land went to them, and Felarya just kept acting normally. It's like killing a shark for attacking a surfer. The surfer went into the shark's world, and the shark was...well, just being a shark. All you can really do is move on and learn from it, since finding the same shark again and killing it is just a waste of time for no real gain.