Fuzzy inference systems have been successfully applied to many real-world applications. Traditional fuzzy inference systems are only applicable to problems with dense rule bases covering the entire problem domains, whilst fuzzy rule interpolation (FRI) works with sparse rule bases that do not cover certain inputs. Thanks to its ability to work with a rule base with less number of rules, FRI approaches have been utilised as a means to reduce system complexity for complex fuzzy models. This is implemented by removing the rules that can be approximated by their neighbours. Most of the existing fuzzy rule base generation and simplification approaches only target dense rule bases for traditional fuzzy inference systems. This paper proposes a new sparse fuzzy rule base generation method to support FRI. In particular, this approach uses curvature values to identify important rules that cannot be accurately approximated by their neighbouring ones for initialising a compact rule base. The initialised rule base is then optimised using an optimisation algorithm by fine-tuning the membership functions of the involved fuzzy sets. Experiments with a simulation model and a real-world application demonstrate the working principle and the actual performance of the proposed system, with results comparable to the traditional methods using rule bases with more rules.