Why does Buck decide to leave the Baxters so abruptly in The Yearling?

I am assuming that you are referring to the incident in
Chapter 24, when Buck comes over the the Baxters' place to invite Penny to join them in
hunting wolves the next morning at dawn. The wolves had been heard howling from neaby
the Forresters' Island, and Buck knows that "with luck, all the remaining wolves might
be killed (and) game also could easily be taken." Appreciative, Penny accepts the
invitation, and asks Buck to spend the night, but Buck says that he must leave. He
explains to Penny that if he does not show up back at home by bed-time, his brothers
will figure that there will be no hunt, and will not be ready early the next
morning.

The other incident to which you might be referring
occurs in Chapter 25. When the hunt was nearly over, the Baxters and Forresters had come
upon the tracks of a big buck deer, but had agreed not to go after it that day. While on
his way home, the buck had run across Penny's track, and Penny, taking advantage of the
opportunity, had killed it. When Lem sees the hide stretched out at the Baxters' place,
he gets angry because he thinks that Penny has betrayed the Forresters, going after the
buck on his own so he could have it for himself. Penny tells Lem the truth, that he came
upon the buck by "pure happen-so," and Buck sticks up for Penny as an honest man. The
Forresters, and Buck in particular, seem distant after that, and when Buck finally comes
by again to tell Penny the wolves have been eradicated, Penny
says,

"I hope
you ain't mixed up in your mind about that buck Lem got so ornery
about."

Buck evasively
says,

"That's
all right. What's one
deer?"

and leaves quickly,
indicating that perhaps he does believe that Penny had been trying to take advantage of
the Forresters by going after the buck on his own, and is understandably angry and
hurt.